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consists  of  exogenous  arrivals  and  endogenous  reentries  from  node  two.  The 
probability  of  a  customer  feeding  back  is  allowed  to  depend  on  the  queue 
length  at  nodes  one  and  two  and  on  the  service  time  of  the  customer  at  node 
one.  ! 

The  system  is  analyzed  by  showing  that  {X  ,  T  }  forms  a  Markov  renewal 

n  n 

process,  where  T  is  the  time  of  the  nth  output  and  X  is  an  ordered  triplet 
n  n 

consisting  of  the  two  queue  lengths  at  time  Tn  and  a  binary  random  variable 

indicating  whether  the  nth  output  departs  or  feeds  back  to  node  two.  The 

semi-Markov  kernel  is  computed  and  conditions  for  {X  ,  T  }  to  be  an  irreducible, 

n  n 

aperiodic  Markov  renewal  process  with  an  infinite  lifetime  are  determined.  A 
necessary  and  sufficient  condition  and  an  easily  checked  sufficient  condition 
for  ergodicily  are  established. 

Using  the  results  about  {X^,  T^},  it  is  shown  that  the  joint  queue  length 
process  is  a  semi-regenerative  process.  The  time-dependent  joint  queue  length 
distribution  is  given.  The  busy  period  of  the  system  is  shown  to  form  a 
renewal  process  and  its  interrenewal  distribution  is  obtained. 

The  customer  flows  generate  point  processes.  It  is  shown  that  the  outputs 
from  node  one,  the  feedbacks  to  node  two  and  the  departures  from  the  network 
are  each  Markov  renewal  point  processes.  It  is  also  shown  under  general 
conditions  that  if  the  departure  process  is  a  renewal  process,  the  departure 
process  must  be  a  Poisson  process.  In  addition,  if  the  departure  process  is  a 
Poisson  process,  either  the  server  at  node  one  is  also  an  exponential  server  or 
there  are  an  uncountable  number  of  initial  distributions  yielding  a  Poisson 
departure  process. 

In  analyzing  the  flow  processes,  a  result  is  derived  which  has  interest 
independent  or  queues  with  feedback.  Consider  a  right-continuous  Markov  process 
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(X(t) :  t  ^  0}  with  left  hand  limits  and  an  invariant  probability  measure  p. 

It  is  shown  that  if  {Tn>  forms  a  non-anticipating  Poisson  point  process  and 
if  {X(Tn~)}  is  a  Markov  chain,  then  is  a  stationary  distribution  for 
(X(Tn~)}.  This  result  implies  that  the  distribution  seen  by  Poisson  arrivals 
to  a  queueing  network  is  the  same  as  the  distribution  at  an  arbitrary  point  in 
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CHAPTER  I 


INTRODUCTION 

In  this  dissertation  we  analyze  a  two  server  queueing  network 
with  feedback.  Chapter  I  provides  an  overview  of  this  subject  and  of 
this  dissertation.  In  the  first  section  of  Chapter  I,  we  supply  some 
background  on  the  subject  of  queueing  networks  and  feedback  in  queueing 
networks  which  provides  motivation  for  analyzing  the  M/G/l  queue  with 
delayed  feedback.  Section  two  contains  an  informal  description  of  the 
M/G/l  queue  with  delayed  feedback.  Section  three  reviews  the  literature 
on  queueing  systems  with  feedback.  Section  four  describes  the  main 
results  of  this  dissertation.  In  section  five  the  organization  of  the 
dissertation  is  explained.  It  contains  a  description  of  the  numbering 
scheme  for  chapters,  theorems,  lemmas,  etc.  and  a  brief  account  of  the 
contents  of  the  remaining  chapters. 

1.  Background  Remarks 

The  field  of  queueing  network  theory  dates  back  at  least  twenty- 
five  years  to  R.  R.  P.  Jackson's  (1954)  paper  on  two  M/M/1  queues  in 
tandem.  Recent  interest  in  modelling  computer  systems  and  computer- 
communications  networks  has  spurred  growth  in  queueing  network  theory. 
Two  different  philosophies  on  the  analysis  of  queueing  networks  have 
evolved.  The  first  approach  analyzes  the  queueing  network  as  a  whole. 
The  second  approach  involves  decomposing  the  network  into  subnetworks 
and  analyzing  each  subnetwork  separately.  In  order  to  do  this,  the 
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departure  process  of  each  subnetwork  must  be  characterized  as  well  as 
the  superposition  and  decomposition  of  flow  processes  in  the  network. 

This  Interest  in  the  flow  processes  has  gradually  led  to  the  realization 
that  queueing  networks  with  feedback  are  fundamentally  different  from 
queueing  networks  without  feedback.  Burke  (1976)  first  exhibited  this 
difference  when  he  showed  that  the  input  process  of  the  M/M/1  queue  with 
instantaneous  feedback  is  not  a  Poisson  process.  Since  all  flow  proces¬ 
ses  in  Jackson  networks  without  feedback  are  Poisson  processes,  the 
introduction  of  feedback  has  fundamentally  altered  the  input  process. 
Subsequently,  Foley  (1977)  proved  that  the  output  process  of  the  M/M/1 
queue  with  instantaneous  feedback  was  not  even  a  renewal  process. 

Despite  the  marked  difference  in  queueing  networks  with  feedback, 
the  literature  contains  only  a  handful  of  theoretical  papers  analyzing 
queues  with  instantaneous  feedback  and  even  fewer  studying  queueing 
networks  with  delayed  feedback.  These  queueing  systems  are  important  as 
evidenced  by  the  many  applied  papers  modelling  systems  as  queues  with 
feedback.  Disney  and  Wyszewianski  (1975)  survey  over  200  papers  which 
model  computer  systems  as  queues  with  feedback. 

Thus  a  detailed  analysis  of  the  M/G/l  queue  with  delayed  feed¬ 
back  will  be  useful  to  practitioners  modelling  real  world  systems  with 
delayed  feedback.  Furthermore,  it  will  provide  insight  into  understand¬ 
ing  more  general  networks. 

2.  Informal  Problem  Description 

A  queue  with  delayed  feedback  consists  of  a  queue  in  which  cus¬ 
tomers  completing  service  either  depart  from  the  system  or  feed  back  to 
a  delay  system.  A  fed  back  customer  is  delayed  for  a  random  length  of 
time  before  reentering  the  queue  (see  fig.  1). 
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Fig.  1.  A  Queue  with  Delayed  Feedback 


We  have  chosen  to  model  the  delay  mechanism  as  a  single  server 
queue  with  exponential  service  times  and  an  infinite  queue  capacity. 

Other  models  are  possible.  Nakamura  (1971)  modelled  the  delay  mechanism 
as  an  infinite  server  queue  with  exponential  service  times.  In 
Nakamura's  system  the  delay  represented  the  time  a  user  of  a  time-sharing 
system  spent  thinking  before  entering  another  command. 

The  M/G/l  queue  with  delayed  feedback  is  a  queue  with  delayed 
feedback  in  which  the  arrival  process  (see  fig.  1)  is  a  Poisson  process 
and  the  lower  server  is  a  general  server.  The  M/G/l  queue  with  delayed 
feedback  as  shown  in  figure  2  is  the  system  analyzed  in  this  paper. 


Fig.  2.  The  M/G/l  Queue  vich  Delayed  Feedback 

We  have  noc  specified  how  a  customer  is  selected  to  feed  back. 
We  will  be  more  specific  in  Chapter  II.  In  general  the  probability  of 
feeding  back  will  be  allowed  to  depend  on  the  queue  lengths  at  both 
servers  and  the  service  time  of  the  customer  completing  service. 
Bernoulli  feedback  is  an  Important  special  case  in  which  each  customer 
feeds  back  with  a  fixed  probability  p  independent  of  the  past  history 
of  the  system.  Note  that  the  M/M/1  queue  with  delayed  Bernoulli  feed¬ 
back  is  a  Jackson  (1957)  queueing  network.  We  assume  throughout  the 
dissertation  that  customers  are  served  using  a  first-come,  first-serve 
queue  discipline  and  that  both  queues  have  an  infinite  capacity  waiting 
area. 

3.  Literature  Review 

In  this  section  we  will  review  the  literature  which  pertains  to 
queueing  systems  with  feedback.  However,  we  will  not  review  the 
enormous  literature  on  classical  queueing  systems  which  fall  under  the 
heading  of  G/G/c  queues.  For  background  literature  in  classical  queue¬ 
ing  theory,  the  reader  is  referred  to  Cohen  (1969)  and  Syski  (1960). 


For  a  review  of  queueing  network  theory,  the  reader  should  consult 
Disney  (1975),  Lemoine  (1977)  or  the  review  in  the  beginning  of  Bozina's 
(1977)  dissertation.  Disney  and  Wysewianski  (1975)  survey  over  200 
papers  which  model  computer  systems  as  queueing  systems  with  feedback. 

The  M/G/l  queue  with  instantaneous  Bernoulli  feedback  has  been 
examined  by  several  authors  including  Takacs  (1963),  Davignon  (1974), 
Davignon  and  Disney  (1976),  Burke  (1976),  Foley  (1977),  and  Disney  and 
McNickle  (1977).  For  a  stationary  system,  Takacs  calculates  the  queue 
length  distribution  and  derives  the  Laplace-Stieltjes  transform  of  the 
total  time  spent  in  the  system  by  a  customer.  Even  in  the  M/M/1  queue 
with  instantaneous  Bernoulli  feedback  the  transform  is  quite  complicated. 
Takacs  was  able  to  determine  the  first  two  moments  of  the  distribution. 
Takacs  analyzed  the  queue  length  process  by  finding  an  equivalent  M/G/l 
queue  without  feedback.  The  equivalent  queue  retains  the  same  queue 
length  characteristics  as  well  as  the  same  departure  process  character¬ 
istics. 

Davignon  and  Disney  (1977)  analyze  the  M/G/l  queue  with  instan¬ 
taneous  state  dependent  feedback.  The  probability  of  a  customer  feeding 
back  after  completing  service  was  allowed  to  depend  on  the  change  in  the 
queue  length  since  the  last  service  completion,  the  length  of  service 
received,  and  whether  the  previous  customer  fed  back  or  departed.  For 
this  system,  the  authors  characterize  the  stationary  queue  length  process, 
output  process,  and  departure  process.  The  busy  period  distribution  is 
also  computed. 

Burke  (1976)  used  the  M/M/1  queue  with  instantaneous  Bernoulli 
feedback  as  a  counterexample  to  the  belief  that  the  random  processes 
representing  the  flows  on  the  arcs  in  Jackson  (1957)  networks  are  Poisson 
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processes.  Many  queueing  theorists  and  applied  researchers  believed 
that  the  £lows  on  the  arcs  in  Jackson  networks  are  Poisson  processes 
(e.g.  Nakamura  (1971)). 

Subsequently  Foley  (1977)  showed  that  the  output  process  in  the 
M/M/1  queue  with  instantaneous  Bernoulli  feedback  in  equilibrium  is  not 
even  a  renewal  process.  This  result  is  surprising  since  even  though  the 
output  process  fails  to  be  a  renewal  process,  the  departure  process, 
which  is  obtained  by  randomly  selecting  points  from  the  output  process 
with  a  fixed  probability  q,  is  a  Poisson  process. 

Disney  and  Hannibalsson  (1977)  analyze  an  M/M/1  queue  with  delay¬ 
ed  Bernoulli  feedback.  The  model  analyzed  is  similar  to  the  M/M/1  queue 
with  delayed  Bernoulli  feedback  as  discussed  in  this  paper  except  that 
each  queue  has  a  finite  waiting  room.  The  steady  state  distribution  of 
the  two  dimensional  Markov  process  (N(t),  M(t)}  is  determined  where  N(t) 
is  the  queue  length  in  the  lower  system  and  M(t)  is  the  nunber  in  the 
delay  system.  An  algorithm  was  developed  to  numerically  compute  the 
steady  state  distribution  for  a  specific  system. 

4.  Main  Results 

Our  first  result  shows  that  the  state  of  the  system  at  output 
points  (see  fig.  1),  (X°,  T°),  is  a  Markov  renewal  process.  T°  is  the 
time  of  the  nth  putput  and  X°  is  an  ordered  triplet  consisting  of  the 
two  queue  lengths  at  time  T°  and  an  indicator  of  whether  the  customer 
departs  or  feeds  back.  This  provided  the  foundation  for  much  of  the 
following  analysis.  The  semi-Markov  kernel  for  (X°,  T°)  is  computed  and 
conditions  for  (X°,  T°)  to  be  an  irreducible  aperiodic  Markov  renewal 
process  with  an  infinite  lifetime  are  determined  based  on  the  governing 
sequences.  The  governing  sequences  are  the  Interarrival  times,  service 
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cine,  at  the  upper  and  lover  queues,  and  the  probability  of  a  customer 
feeding  back,  A  condition  for  (X°,  T°)  to  be  ergodic  is  determined 
which  involves  the  invariant  measure  for  the  Markov  chain  X°.  Since  in 
general  the  invariant  measure  is  not  explicitly  known,  an  easily 
calculated  sufficient  condition  is  also  determined  by  applying  one  of 
Foster's  criteria  (cf.  Cohen  (1969)). 

In  the  following  section.  III. 2  we  examine  the  time  dependent 
queue  length  process,  Z  *  {Zt>.  Z  is  shown  to  be  a  semi-regenerative 
process  with  Imbedded  Markov  renewal  process  (X°,  T°) .  The  probability 

P(Zt  *  (J J 2^  I  xo  “  ’  T0  "  ^  is  determined  and  the  Unit¬ 

ing  distribution  of  Z  is  analyzed. 

In  III. 3  the  busy  cycle  is  shown  to  be  a  renewal  process.  The 
busy  cycle  is  the  time  between  two  successive  returns  to  a  completely 
empty  system.  Since  the  busy  cycle  is  a  renewal  process,  we  need  only 
describe  the  Interrenewal  distribution  to  characterize  the  process.  The 
interrenewal  distribution  is  found  by  using  the  fact  that  successive 
returns  to  an  empty  system  correspond  to  a  Markov  renewal  process 
obtained  by  filtering  (X°,  T°). 

The  point  processes  generated  by  customer  flows  are  analyzed  in 
III. 4.  The  output  process,  departure  process  and  feedback  process  are 
shown  to  be  Markov  renewal  processes.  In  particular  we  are  concerned 
with  determining  conditions  for  the  departure  process  to  be  a  renewal 
process.  We  show  under  general  conditions  that  if  the  departure  process 
is  a  renewal  process  it  must  be  a  Poisson  process.  A  conjecture  is 
stated  which,  if  verified,  implies  that  in  order  to  have  a  renewal 
departure  process,  the  lower  server  must  be  an  exponential  server. 

In  Chapter  IV  we  prove  some  results  which  were  used  in  Chapter 
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III  and  also  have  interest  independent  o£  queues  with  delayed  feedback. 
We  assume  that  we  have  a  right  continuous  Markov  process  {X(t):  t  >_  0} 
with  left  limits  and  an  invariant  probability  measure  v*.  We  show  that 
if  a  sequence  of  stopping  times  {Tq}  forms  a  non-anticipating  Poisson 
point  process  and  if  (X(Tq-) }  is  a  Markov  chain,  then  p  is  a  stationary 
distribution  for  {X(Tq-) }.  The  sequence  of  stopping  times  is  non- 
anticipating  if  the  forward  recurrence  time  until  the  next  stopping  time 
after  t  is  Independent  of  X^.  This  result  can  be  used  to  show  that  the 
distribution  seen  by  Poisson  arrivals  to  a  queueing  network  is  the  same 
as  the  distribution  at  an  arbitrary  point  in  time. 

S.  Organization 

This  dissertation  is  divided  into  five  chapters  and  one  appendix. 
Chapters  are  assigned  a  Roman  numeral.  Each  chapter  is  subdivided  into 
sections  which  are  assigned  an  Arabic  numeral.  Hence,  the  third  section 
of  Chapter  IV  is  referred  to  as  IV. 3.  Within  each  chapter;  definitions, 
lemnas,  propositions  and  theorems  are  assigned  a  name,  n^.n^,  where  n^ 
is  the  section  number  and  n^  is  assigned  consecutively  within  the 
section.  Within  a  chapter,  the  Roman  numeral  is  suppressed.  Hence  in 
Chapter  III  we  might  refer  to  Lemma  2.1,  but  outside  of  Chapter  III  we 
would  refer  to  Lemma  III. 2.1.  Equations,  whenever  needed,  are  refer¬ 
enced  by  an  Arabic  numeral  in  parentheses  to  the  right  of  the  equation 
and  numbered  consecutively  within  a  chapter.  Figures  are  numbered 
consecutively  throughout  the  dissertation.  Bibliographic  references 
consist  of  the  author's  name  followed  by  the  publication  date  enclosed 
in  parentheses. 

In  Chapter  II  we  normally  describe  the  M/G/l  queue  with  delayed 
feedback  in  terms  of  the  governing  sequences.  The  governing  sequences 


are  the  arrivals,  service  times  at  the  upper  and  lower  queues,  and  the 
probability  of  a  customer  feeding  back.  The  processes  of  interest,  e.g. 
queue  length  process,  are  defined. 

Chapter  III  contains  the  analysis  of  the  M/G/l  queue  with 
delayed  feedback.  The  system  at  output  points  is  analyzed  as  a  Harkov 
renewal  process.  The  time  dependent  queue  length  process  is  described 
and  its  limiting  behavior  studied.  The  busy  cycle  is  shown  to  be  a 
renewal  process  and  an  expression  of  the  interrenewal  distribution  is 
obtained . 

Chapter  IV  contains  results  for  more  general  systems  than  the 
M/G/l  queue  with  delayed  feedback  and  were  used  in  Chapter  III.  We 
show  that  under  certain  conditions  a  Markov  process  and  a  Markov  chain 
imbedded  In  the  Markov  process  have  the  same  stationary  distribution. 
These  results  are  useful  in  areas  other  than  queues  with  delayed 
feedback.  In  Chapter  V,  we  summarize  the  results  in  this  dissertation 
and  several  areas  of  further  research  are  discussed. 


CHAPTER  II 


FORMAL  PROBLEM  DESCRIPTION 

In  Chapter  II,  we  formally  discuss  the  problem  to  be  studied. 
Section  one  contains  a  description  of  the  physical  structure  of  the 
queueing  network.  In  section  two,  the  governing  sequences  and  assump¬ 
tions  about  the  governing  sequences  are  listed.  Section  three  introduces 
notation  for  the  processes  of  interest  (e.g.  queue  length  process),  which 
are  determined  by  the  governing  sequences  and  the  structure  of  the  net¬ 
work. 

1.  Queueing  Network  Structure 

The  queueing  network  contains  two  servers  as  shown  in  figure  3. 
The  lower  queue  will  be  denoted  Q^,  and  the  upper  queue,  Q2»  and  Q2 
are  FIFO  queues  with  infinite  queue  capacities.  All  external  arrivals 
enter  Q^.  After  waiting  and  being  serviced  at  Q^,  the  customer  outputs. 
Some  outputs  depart  from  the  system  and  the  remainder  feed  back  to  Q2. 
After  waiting  and  being  serviced  at  Q2,  the  customer  reenters  (see 
figure  3) . 
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Fig.  3.  Structure  of  the  Queueing  Network 


2.  Governing  Sequences  of  Random  Variables 

The  governing  sequences  are  the  basic  random  variables  (e.g. 
service  times)  from  which  the  processes  of  interest  (e.g.  queue  length) 
are  defined.  Let  {A  :  n-1,2,...},  (S^:  n-1,2,...}, 

II  tl  • 

(2) 

(Sn  :  n-1,2,...},  and  {Yn:  n-1,2,...}  be  non-negative,  real  valued 

random  variables  on  a  complete  probability  space  (fl,  &,P) .  A  will 

n 

denote  the  nth  in ter arrival  time  to  the  system;  ,  the  nth  service 
(2) 

time  at  Q. ;  S  ,  the  nth  service  time  at  Q0 ;  and  Y  will  be  either  0 
•in  i  n 

or  1  depending  on  whether  the  nth  customer  completing  service  at 

departs  or  feeds  back,  respectively.  Note  that  the  nth  arrival  does 

not  necessarily  receive  the  nth  service  time  or  the  nth  state  of  the 

decomposition  switch.  (A  },  {S^},  and  {S^}  are  each  i.i.d.  sequences 

n  n  n 

of  random  variables  and,  furthermore  are  independent  sequences  with  the 
following  properties: 

-Xt 


P{A_  <  t}  ■  1  -  e 


0  <  X  <  ®, 


t  >  0, 


Thus,  the  arrival  times  form  a  Poisson  process.  has  a  general 

server.  has  an  exponential  server.  Any  sample  path  u  e  ft  determines 

four  sequences  of  random  variables;  {A  (w)},  the  interarrival  times; 

n 

the  service  times  at  Q. ;  {S^(w)},  the  service  times  at  Q„; 
n  in  i. 

and  {Yq(w)},  the  states  of  the  decomposition  switch.  Y^  will  be 

allowed  to  depend  on  {A  },  {S^},  {S^},  and  {Y  }  in  a  limited 

n  n  n  n 

fashion.  We  will  describe  the  probabilistic  structure  of  Y^  in  section 
three . 

3.  Processes  of  Interest 

We  begin  section  three  with  a  brief  introduction  to  point  proces¬ 
ses.  Most  of  the  following  is  taken  from  Daley  and  Vere-Jones  (1972). 

Let  X  be  any  complete  separable  o-compact  metric  space.  Let  JV  be  the 
space  of  all  non-negative  integer-valued  measures  £(•)  defined  on  the 
o-algebra  3B(X)  of  all  Borel  sets  of  the  state  space  X,  with  N(A)  <  • 
for  all  bounded  A  c  ^8(.X) .  Let  be  the  o-algebra  of  subsets  of  jV 
generated  by  all  sets  of  the  form  (K:  N(A)  <_  k),  (k  e  {0,1,2,...}, 

A  e  #(X». 

Definition  3.1.  A  general  (stochastic)  point  process  is  a 
measurable  mapping  from  a  probability  space  (0,  «^*\p)  into  (  jV, 

Definition  3.2.  A  point  process  is  a  general  point  process  with 
state  space  X  ■  *  (0,®]  . 

Definition  3.3.  A  general  point  process  with  state  space 
X  *  Y  x  is  called  a  marked  point  process  and  Y  is  referred  to  as  the 
mark  space . 
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Marked  point  processes  are  sometimes  called  labelled  point 
processes  or  point  processes  with  an  ancillary  variable.  At  time  0, 

1^  customers  are  in  and  i^  customers  are  in  Q2>  Six  point  processes 
will  be  used  to  represent  the  flows  of  customers  in  the  system. 

N3  will  be  the  point  process  representing  the  arrival  process. 

is  a  measurable  mapping  from  (ft,  ^,P)  into  (  »+,  For 

any  u>  e  ft  ,  Nt(w)  is  the  number  of  arrivals  in  the  interval  (0,t],  l.e. 

■  Na(0,t].  Similarly,  N*  will  represent  the  input  process  to  Q^,  N° 
will  represent  the  output  process  from  Q  ,  will  represent  the 
departure  process  from  the  system,  will  represent  the  feedback  process 
to  Q2,  and  Nr  will  represent  the  reentry  process  from  Q2  to  These 

six  point  processes  are  determined  by  the  initial  conditions  and  the 
governing  sequences.  It  is  easy  to  see  that 

N*  «  Na  +  Nj;  and  N°  -  +  N*. 

From  Na  we  can  determine  Ta,  the  time  of  the  nth  arrival.  Let  Ta *  {Ta}. 
t  n  n 

Similarly  we  can  determine  T*,  T°,  Td,  T  ,  and  Tr. 

In  the  following  theorem  we  construct  the  point  processes  re¬ 


presenting  the  customer  flows  and  show  that  the  point  processes  are 
measurable  functions  defined  on  (ft. 

Theorem  3.4.  Na,  hJ,  N°,  kJ,  nJ,  kJ,  ij.  tJ.  T°,  T*.  T*  and  T° 
are  measurable  functions  from  (ft,  P)  into  (  K+,  #(*+)>. 

Proof  by  induction.  Initially  there  are  i^  customers  in  and 
i2  customers  in  Q2.  Define 

To  ■  T?  •  T5  *  TS  ■  To  ■  ■  To  •  °- 

v  -  v .  v  -  v  -  v  -  v  -  0, 

n  n  n  n  n  n 

0„a  0..i  0  o  0.  d  0Mf  0Mr 

t  t  L  t  L  t 


0. 
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d  3 

We  will  construct  a  sequence  of  measurable  functions  T  t  T  as  i  +  «. 

n  u 

Similarly  for  Ti,  T°,  Td,  Tf,  and  Tr,  there  will  be  a  sequence  of 
n  n  q  n  n 

measurable  functions  converging  polntwlse  to  each  of  the  functions.  Let 

T  denote  the  time  of  the  nth  event.  An  event  consists  of  a  change  in 
n 

either  one  or  both  queue  lengths.  Thus  an  event  corresponds  to  either 
an  input  or  an  output.  Let  Z(Tg)  *  (i^,^).  Z(Tn)  will  denote  the 
queue  length  just  after  time  Tn>  the  time  of  the  nth  event.  Our  proof 
is  by  induction.  We  assume  that  T^,  ^j*  ’  ^t’ 

‘‘N* ,  *11°,  Z(Tk)  are  measurable  for  all  k  n.  Then  we 

.ho.  Chat  1^,  "+1T*.  “+lrJ,  "+lT°,  "+1tJ,  »+1^, 

n+1N°,  n+1H^,  n+LN^,  n+LN^,  and  2(1^)  ace  measurable.  We  will 
sometimes  write  °Na(t)  for  **11*,  and  similarly  for  the  other  counting 
processes.  Now  Tn+1  can  be  expressed  as 

Tn+l“  X(0,»)  x  (0,®)(Z(Tn)){min^nTnNa(T  ^  +  ^ 

n  n 

t  )  V(nT°  ) 

(0,op)x[0,«)  (  V(T  }  )(  V(T  )  V(T)+1 


n 


n 


+  ',0,a(0..)«<\o(I)»<\l(nIo  )  +  l  +  V(T>  +  l 

n  V’CT  ^  n 

n 

T  (7(nT*  )  Wn'rr  ) 

[0,«)  x  (0,«)  "Nr(T  )  V(T  )  V(T  )+l 

n  n  n 

♦I[0,.)*{0)«‘\r(I))><,'IV(V  )+1  +  SV(I)  +  1>l> 

n  V(T  )  n 

n 

+  I,nw  (n  a(Z(T  )){min[V  +A 

{0}  x  (0,  )  n  V(T  )  V(T  )  +  l 

n  n 

I[0,«)x(0.»)(Z(nTnNr(T  }))(\r(T  )  +  SV(T  ^ 

n  n  n 


.(1) 


), 
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♦V->x<0><‘<\I(T)>><%,f(V  )+1  +  *£om>» 

n  V(T  ) 

n 


■/n  ^  /ni(Z(T  )){min[nTa  +A 

<0,-)xf0)  „  V(T)  V(T)  +  1 


V-)x(0..,<2<\o(t  ,»<\o (i,+SV(i)+1> 

n  n  n 


,>><%,  1(V  )  +  l  +  sV(T)+1),} 

n  V(i  ) 

n 


+  r (Z(T  ))[DTa  + A  ]. 

{0}  x  {0}  n  ^(T  )  ^(T  )+1 

n  n 


A  little  explanation  of  the  previous  equation  is  due.  The  equation  is 
divided  into  four  terms  depending  on  the  queue  length  at  time  T^.  In 
the  first  term  the  queue  length  at  both  queues  is  strictly  positive, 
hence  there  is  a  customer  in  service  at  both  queues.  Consequently,  the 
time  of  the  next  event  is  the  minimum  of  three  random  variables:  the 
time  until  the  next  arrival,  the  time  until  the  next  output,  and  the 
time  until  the  next  reentry.  The  time  of  the  next  arrival  is  the  time 
of  the  last  arrival  plus  the  next  interarrival  time.  The  time  until  the 
next  output  must  be  broken  into  two  cases.  In  the  first  case,  the  queue 
length  at  at  the  time  of  the  last  output  was  greater  than  zero.  Hence 
the  time  of  the  next  output  is  the  time  of  the  last  output  plus  the  next 
service  time.  In  the  second  case  at  the  time  of  the  last  output, 
became  idle.  Hence  the  time  until  the  next  output  is  the  time  until  the 
first  input  after  it  became  idle  plus  the  next  service  time.  Note  that 
an  input  must  have  occurred  by  time  Tfl  since  has  a  positive  queue 
length  at  time  T^.  The  time  until  the  next  reentry  is  also  divided  into 


two  cases  analogously  to  the  time  until  the  next  output  except  that  we 
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condition  on  whether  Qj  is  busy  or  Idle  at  the  time  of  the  last  reentry. 

Nov  T  , ,  is  measurable  since  T  . ,  is  a  combination  of  sums,  products, 
n+1  n+1 

mins  and  compositions  of  functions  which  are  measurable  either  by  the 
inductive  hypothesis  or  by  being  a  member  of  one  of  the  governing 


sequences.  We  have  implicitly  assumed  chat  ^T' 


is  the  time  of  the 


v<v 

last  arrival  in  (0,  T  )  and  similarly  for  nT°  ,  etc. .  This  will 

n  V(T  ) 

n 

follow  from  the  following  recursive  definition  of  and  nN^.  First 

note  that  a.s.,  since  in  every  case  the  time  until  the  next 

event  is  the  sum  of  some  past  time  plus  an  independent  random  variable, 

which  is  exponentially  distributed  in  all  cases  except  when  the  random 

variable  is  S^.  We  know  that  an  event  has  occurred.  Now  we  need  to 
n 

decide  whether  it  was  an  arrival,  departure,  feedback,  or  reentry. 

Tn+^  has  the  following  form 


Define 


Vl  '  V-)  ,  fo’  £r>> 


+  I(0}x(0,.)(Z(V)(”1“[f.'  £r» 
+  I<0,-),{0><Z<V)("I”t£a-  £o» 
+  I(0>*(0>Z<V>£.  ' 


[I(0,«)  x  (0,-)<Z(V)I(0,«)<£o'£a,l(0,-)(£r‘£a> 
+  I<0>,,(0,-)(Z(t„>>I<0..><£r-f.>  + 


+  I(0,«)x{0}(Z{Tn))I(01»)(fo‘fa) 


+  1{0}x {0}(Z(Tn))]Tn+l 


"*X  ■  X  + 

+  I(0,-)x{0)(Z<V>I(0,-)(£a'fo)1(1"Yiy>(  )+1)Tn+l 

n 

^X-X*  wv<v> 

•  ‘V-)  x  (0,-)<Z<V>I(0.-)('a-£o)I(0,.)(£r-£o) 

+  I(0,-)  x  (0)(Z(Tn))I(0,-)(£a"fo>1To+l 

”+X  *  X  +  I(k-l)(",,£<V> 

'  'I(0..)x(0..)<Z<Tn»I(0,-)<£a-£o>I(0,.)<£r-£o> 

+  I(0,-)x{0)(ZCV)I(0,.)Ka"fo)IYa11o(I  ,  +  ^n+l 

n 

^X  ■  X +  i(k-n<V(v) 

•  tI(0,-)x(0.-)<Z(In»I(0,.)<f.-£r>I(0,.)(£o-£r) 

•  £{0}  x  (0,»)(Z(V)I(0,-)<fa'fc)!Ta+l 

""X  ■  X  +  I{k-l><V(Tn» 

•  IIC0.-)x(0..)(Z(Ia»(1-I(0..)(£a-£o>I(0..)<£r-£o») 

+  I(0),(0..)lZ(In»  + 
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+  I(0.-)x{0)(Z(T«»I(0>.)(fo-f.> 

tI(OhlOI<!,V)lV 

Thus  n+1T®  is  the  time  of  the  kth  arrival  if  it  is  one  of  the  first  n  +  1 
events.  Otherwise,  «  0.  The  same  observation  is  true  about 


n+l„i  n+lmo  n+1  d  n+Lf 


V 


T 

V 


and  n+^T^.  Now  define 


/n+1  a. 

(0,t]1  V* 

“+V"> 

k 

/n+i-o. 

(O.tr  V* 

(n+1Tf) 

(O.tr  V* 

Thu9,  n+V  (t)  is  the  number  of  arrivals  in  the  interval  (0,t)  which  are 
also  one  of  the  first  n+1  events.  The  same  holds  for  the  other  counting 
processes.  Now, 

Z(W •  -”+1"r(Vi»- 

and  we  have  completed  the  induction.  Each  of  the  functions  defined 

above  are  measurable  since  we  have  expressed  them  as  sums,  products  or 

compositions  of  measurable  functions.  Now  as  n  +  « 

n^a  ^  rpa  riji  ,  ^.i  n^o  ^  T^ 

Ak  k’  Ak  Ak*  Ak  k* 


X  +  if,  nT^  t  T*  V;  +  Tv, 

k  k’  k  k’  k  k’ 

X +  N?’  X  +  Nt’  X  *  Nt* 

X +  Nt’  X  +  X  X +  X 


Our  proof  is  completed  since  if  {f^}  are  measurable  and  ffl  -*•  f, 
then  f  is  measurable.  <3 

Let  K  -  {0,1,2,...},  F  -  K  x  K  and  (F,  be  the  discrete 


f 


topological  space.  The  stochastic  process  Z  -  { Z  :  t  _>  0}  will  be  the 


joint  queue  length  process,  where 


Zq(oj)  -  (ilt  i2) 


Zt(o>)  -  (iL  +  N^(u)  -  N°(w),  i2  +  N*(w)  -  N*(u>)),  t  >  0. 

If  Zt  -  (j^,  J2)  then  at  time  t,  has  customers  and  Q2  has  j2 
customers:  Zfc  is  a  measurable  function  from  to  (F,^),  and  is 

a  right  continuous  function  of  t  having  left  hand  limits  a.e.. 

We  now  characterize  the  probabilistic  structure  of  {Yq},  the 
decomposition  switch.  We  assume  that 

p<Vi "  k  I  V  c  i  I»+i) 

-  P{Vl  ’  k  I  V  , -  ■  «X-  j2>>  S£l  ■  *} 

n+l 


h^j^  j2>  x)  k  -  0,  1, 


otherwise. 


Note  that  •••»*„  is  determined  by  {Z£;  t  T°+1}  • 
Bernoulli  feedback  is  a  special  case  in  which 


\(Jr  j2-  x) 


if  k  -  1, 


1  -  p  if  k  ■  0. 


With  Bernoulli  feedback  {Y  }  is  a  i.i.d.  sequence  of  Bernoulli  random 

n 

variables  and  independent  of  the  other  governing  sequences. 

In  much  of  our  analysis,  we  will  be  examining  the  system  at  out¬ 
put  points.  Hence,  X®  will  represent  the  state  of  the  system  at  time 

T°.  Let  E  -M  xW  x  {0,1},  (E,  $  )  be  the  discrete  topological  space, 
n 

X®  -  (ZQ,  0)  and  X®  -  (ZTo_  -  (1,  0),  Yn>.  Thus  X®  is  an  ordered  triple 


and  if  X  -  (i, ,  i„,  i,)  then  lust  after  the  nth  output  from  Q, ,  i, 
n  l  i  j  i.  i 

customers  are  in  Q^,  1 ^  customers  are  in  (excluding  the  nth  output 

should  it  feedback)  and  is  0  or  1  depending  on  whether  the  nth  output 

departs  or  feeds  back,  respectively.  Let  X°  *  {X°}.  (X°,  T°)  will 

denote  {(X°,  T°) }  where  X  is  the  state  at  the  nth  output  and  T°  is  the 
n  n  n  n 


time  of  the  nth  output. 


CHAPTER  III 


ANALYSIS 

la  Chapter  III  ve  study  the  processes  of  Interest  as  defined  in 
section  three  of  Chapter  II.  Section  one  analyzes  (X°,  T°),  the  system 
embedded  at  outputs.  We  show  that  (X°,  T°)  is  a  Markov  renewal  process 
on  a  countable  state  space.  This  allows  us,  in  section  two,  to  analyze 
Z,  the  time  dependent  queue  length  process,  as  a  semi-regenerative 
process.  Section  three  analyzes  the  busy  cycle  of  the  system.  Section 
four  contains  the  analysis  of  the  point  processes  representing  the  flows 
in  the  network. 


1.  The  System  at  Output  Points 

Recall  that  T°  is  the  time  of  the  nth  output  from  Q^,  and  If 
X°  «  (i^,  ij,  ij),  then  at  time  T°-,  i^  +  1  customers  are  in  Q^,  ^ 
customers  are  in  Q^,  and  i^  is  0  or  1  depending  on  whether  the  nth 
output  departs  or  feeds  back,  respectively.  Our  first  theorem  charac¬ 
terizes  the  structure  of  (X°,  T°). 

Theorem  1,1.  (X°,  T°)  is  a  Markov  renewal  process  with  state 

space  E  ■  W  x  W  x  {0,  1). 

Proof.  We  need  to  show  that 


P<*n-H  •  Ul.J2.53).  Tn+1 

*  p(Cj  ■  1 


•  T°  T° } 

’  o . V 

(i^ij.ij)} 
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for  all  (J1,j2tj3)  e  E,  (i^i^i^  e  E,  n  e  » ,  and  t  _>  0.  Let 

0  . .  -  T°  -  T°.  If  i,  >  0,  0  -  S(}J.  If  i,  -  0,  then 

n+1  n+1  n  1  n+1  n+1  1 

0  t ^  +1.  If  i,  +  i^  ■  0,  I  is  an  exponential  random  variable 


2  3 


with  parameter  X.  If  i2  +  i^  >  0,  I  is  an  exponential  random  variable 
with  parameter  (a  +  X).  Now  the  number  of  arrivals  to  and  number  of 
departures  from  Q2  depend  only  on  X°  and  0p ( Since  Y  ^  depends  only 
on  »°d  Sn+1*  ^X°*  T°^  1148  C^e  desired  property.  ° 

Let  Q  denote  the  semi-Markov  kernel  over  E,  that  is 


Vc)  ‘  p{Ci  ■  >•  Ci  -C'lC11 


where  i  -  (i^i^ij),  j  «  (j1,j2,j3>,  and  i,  j  e  E. 


Lemma  1.2 


ft 

j0 


(i2-j2,x)M*(j1-i1+l+j2-i2,x)G(dx) 


if  i.  >  0,  i,  «  0,  -  iL  +  1  >  i,  -  j,  >  0, 


j 


(t-s)ds,  m*  (1,0,  0)  » 


2  2 


if  i.  -  0,  i,  *  0,  i_  *  0,  j.  >  0,  j,  *  0, 


V*) 


J ['(*  +  «0e-(i  +  a)s  Q.jlt-,)  *  ±  d,. 

m  *  (1,  i2,  0),  n  ■  (1,  i2~  1,  0)  , 


if  ix  -  0,  i2  >  0,  i3  -  0, 


Q  (t),  m  -  <ir  i2+l,  0), 


if  i3  -  1, 


0  otherwise , 
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where 


e"yx(ux)^/j!  j  <  k. 


< 


(yx)1/!! 


j  * 


0  otherwise. 


Proof. 

Case  1.  i.  >  0,  i„  ■  0.  Since  i,  >0.  T°.„  -  T°  ■  S^?,  hence 

-  13  1  n+1  n  n+1’ 

it  has  distribution  G(x).  i.  -  j.  customers  must  leave  Q?. 

i2 

M  (i*-1-fx)  is  the  probability  of  this  event  occurring  in  time  x. 

O  cl 

j^-i^  +  1  customers  must  enter  Q^.  Since  i2  -  J2  come  from  Q2, 

-  i^  + 1  +  J2  -  i2  must  arrive  from  the  outside.  -  i^  +  1  +  j2  -  i2,  x) 

is  the  probability  of  this  event  occurring  in  time  x.  Clearly 

jl-il  +  i  >  i2-j2  >  °. 

Case  2.  i^  «  i2  ■  i^  ■  0.  Since  the  system  is  empty,  the  system 
waits  an  exponentially  distributed  length  of  time,  s,  for  an  arrival. 

The  probability  of  now  transferring  from  (1,  0,  0)  to  j  in  time  t  -  s  is 
given  in  case  1. 

Case  3.  i^  *  0,  i2  >  0,  i^  *  0.  Since  i^  *  0  the  system  waits 
for  an  exponential  length  of  time,  s,  until  either  an  arrival  occurs  or 
a  customer  departs  Q2<  An  arrival  occurs  first  with  probability  X/ (\+a) 
and  then  the  probability  of  transferring  from  (1,  ij,  0)  to  j  is  given  in 
case  1.  Similarly  an  output  from  Q2  occurs  first  with  probability 
a/(A+a)  and  then  the  probability  of  transferring  from  (1,  i2~l,  0)  to  j 
is  given  in  case  1. 

Case  4.  i^  ■  1.  The  probability  of  transferring  from  (i^,  i2,  1) 
to  j  is  the  same  as  the  probability  of  transferring  from  (i^,  i2+l,  0) 
to  j .  •  n 
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Hereafter,  we  denote  P{A|X°  ■  i}  as  P^(A).  Let 
Qn(i»  j,  t)  -  PjxJ—  j,  T°  <  t} 

and 


where  Qn(t)  is  the  matrix  with  Qn(i,jit)  as  its  i,  jth  element,  and  R(t) 
is  the  Markov  renewal  kernel  with  elements  R(i,j,t). 


Q(“)  -  lim  Q(t) 

t-H» 

is  the  one  step  transition  matrix  for  the  Markov  chain  X°  (see  Qinlar 


(1975;  p.  314)). 


Criterion  1.3.  (X°,  T°)  is  an  irreducible  Markov  renewal  process 

with  state  space  E  if  and  only  if 


is  not  reachable.  Similarly, 

J  hgC^,  j2,x)  G(dx)  -  1. 


J2>  x)  G(dx)  <  1,  Ji*  e  ** * 

j2,  x)  G(dx)  *  0  then  the  state  (j^,  j2> 
(j^,  J2»  D  is  not  reachable  if 
If  (1)  holds,  then  there  is  a  positive 


(1) 

0) 


probability  of  transferring  to  (i^+1»  i2*  0)*  (ij“l*  i2»  °)  *  i2"^» 

or  (i^,  i2,  1)  from  (i^,  ij,  0)  in  one  step.  Similarly  there  is  a 
positive  probability  of  transferring  to  (i^,  i2+l,  0)  or  (i^,  i2»  0)  from 
(i^,  i2>  1).  By  using  a  finite  number  of  these  steps,  there  is  a  positive 
probability  of  reaching  any  state  from  any  other  state.  O 

Theorem  1.4.  (X°,  T°)  is  aperiodic. 

Proof.  First,  some  state  (0,-,-)  is  reachable,  since  in  any 
state  (i^’,0,  ij^  >  0,  there  is  a  positive  probability  of  entering  some 
state  (ij-1,*,*)* 
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Hence,  we  need  only  show  that  the  distribution  of  the  time 
between  returns  to  (0,  i^)  is  aperiodic.  But  this  follows  since 

the  inter-return  time  is  composed  of  a  random  number  of  service  times 
and  at  least  one  idle  time.  The  idle  time  is  exponentially  distributed, 
hence  the  inter-return  time  cannot  be  arithmetic  with  a  fixed  span  6. 

Let  L  be  the  lifetime  of  (X°,  T°) .  That  is. 


L  -  sup  T  . 
n  n 

The  following  lemma  shows  that  L  is  almost  surely  infinite. 
Lemma  1.7. 


L  =  sup  T  **  «°,  a.s. . 
n  n 


Proof .  Since 


T°  >_  +•••  +  S^', 


.(1) 

Jn 


we  have 


r  „  AD  .  c(l)  .  .  oCl) 

L  >  S,  +  S~  +• • •+  5  . 

—  1  2  n 


By  the  strong  law  of  large  numbers 

.(1) 


lim 

n-*® 


s 


-  H.  E[s.(1)]  a.s.. 
n  1 


Hence 


**•  00  a.s.,  implying  L  *  a.s.. 


In  the  remaining  chapters,  we  will  make  use  of  our  results  about 
(X°,  T°)  in  order  to  analyze  other  processes  of  interest. 


2.  The  Time  Dependent  Queue  Length  Process 

Z  was  defined  in  II. 3  to  be  the  joint  queue  length  process.  If 
Z^.  *  (i^,  ,  then  at  time  t,  contains  i^  customers  and  contains 


Proof .  From  II. 3,  we  know  that  Z  »  {Zt  >  0}  is  a  stochastic 
process  with  the  discrete  topological  space  (F,S))  as  its  state  space, 
that  Zt  is  a  right  continuous  function  of  t  having  left  hand  limits 
a.e..  From  section  one,  we  know  that  (X°,  T°)  is  a  Markov  renewal 
process  with  an  infinite  lifetime.  Clearly,  {X°  »  i}  c  u  <_  T°) . 

This  combined  with  lemmas  2.1  and  2.2  shows  that  Z  is  a  semi-regenerative 


process. 


O 
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Lemma  2.4.  Define  K£(i,  j)  -  P^Z^.  -  j,  T°  >  t},  where 
i  -  (ir  i2,  i3)  e  E  and  j  -  (j^  j.,)  e  P.  Then  Kt(i,  j)  ia 
+  '  - 

Ma  J(i2  +  i3-j2,  t)Mx(j1-i1  +  j2-i2-i3,  t)[l-G(t)] 


if  i  >  0 


-(X+a)  t 


if  i  -  (0,  i2,  i3),  j  «  (0,  i2+  i3),  12  +  i3> 


-Xt 


f 


if  i  -  (0,  0,  0),  j  -  (0,  0), 
-x(t  -  s) 


Kta,  J)-  S  It  1.(0,  0,0),  J  -  (Jj,  0),  >  0, 


I 


(X+o)e‘(X+a)s[l-F(t-s)] 
i.  +  i 


X+a  Ma  (i2  +  i3~J2’  C  “  s)MX(jl +  j2  “  i2  "  i3~1,  *  “  s) 

- 


X+a  a 

,  io  +  i,-l 
X+a  a 


(i2  +  i3-  1- i2,  t-s) 


ds 


where  M^(j,x) 


Mx(ji  + j2-  i2"  i3»  t-s) 

^  if  i  -  (0,  i2,  i3),  j-(j1,j2),  i2  +  i3  >  0, 

e"UX(yx)J/j!  j  <  k, 

^"je  UX(iix)i/i!  j  ■  k. 


-  < 


i-k 

0  otherwise. 


Kt(i,  j)  describes  the  behavior  of  Z  between  semi- regeneration 
points  (i.e.  output  points). 


mm* PWl 


Proof.  Since 


Pi{Zt  “  (jl’  j2} *  T°  >  t}  "  pi <T°  >  c>pi{zt  -  (jr  J2)  I  T1  >  t}> 

we  first  determine  the  distribution  of  T°.  Then  conditioned  on  T°  >  t, 
the  event  Z£  »  (j^,  j^)  will  be  realized  if  the  number  of  people  in  Q2 
decreases  from  i^  +  i^  to  an<*  num^er  of  people  in  increases 
from  i^  to  O 

Let  Pt(i,  A)  ■  P^{Zt  e  A).  By  combining  the  results  of  section 
one  with  Theorem  2.3  and  Lemma  2.4,  we  are  able  to  obtain  the  time 
dependent  solution  to  the  queue  length  process. 

Theorem  2.5.  For  any  i  ■  (i^,  i 2,  i3)  e  E  and  j  -  (j^,  j2>  e  F, 

P.(i,  3)  *  E  I  R(i,  k,  ds)K  (k,  j). 

C  kcE  JO  Z  '  S 

Proof.  Since  Z  is  a  semi-regenerative  process  with  an  embedded 
Markov  renewal  process  (X°,  T°) ,  the  theorem  follows  directly  from 
Theorem  10.6.8  of  Cinlar  (1975;  p.  346).  ° 

To  complete  the  analysis  of  the  queue  length  process,  we  will 
examine  the  limiting  distribution  of  as  t  +  °°.  First,  let  v  be  an 
invariant  measure  for  the  Markov  chain  X°  and  m(j)  =  E^[T°]  <  ®. 

Theorem  2.6.  If  (X°,  T°)  is  an  irreducible  aperiodic  recurrent 
process  and  vm  <  ",  then 

lim  Pt(i,  A)  -  ~  ]T]v(j)  /  Kt(j,A)dt. 
t-*»  j  JO 

Remark:  In  section  one,  we  analyzed  conditions  for  (X°,  T°)  to 
be  irreducible  and  aperiodic. 

Proof.  The  result  follows  directly  from  Theorem  10.6.12  and  from 
the  proof  of  Theorem  10.7.5  case  (b)  of  ?inlar  (1975;  p.  347,  p.  351).  ° 
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Recall  that  hg(j^,  j2»  x)  is  the  probability  of  a  customer 
departing  given  that  his  service  time  was  x  and  that  he  vlll  leave 
behind  customers  in  and  customers  in  Qg.  When 

hg ( j ^ »  j2»  *)  m  ^2^ 

we  can  obtain  some  easily  computed  sufficient  conditions  for  vm  <  ®. 

Let  1/p  denote  the  expected  service  time  of  a  customer.  Since 
1/p  <  <  1/p  +  1/A  for  all  i,  vm  <  ®  if  I  <  ®.  If  X°  is  an 

irreducible  aperiodic  Markov  chain  this  is  equivalent  to  showing  that  X° 
is  ergodlc. 

Theorem  2.7.  X°  is  ergodic  if  X°  is  an  irreducible  aperiodic 
process  and  if  for  all  but  finitely  many 


yh0^1’  -*2^ 

and 

^0»1(j1.J2)  +  */u) 

a(hg(jl» J2)  ~  */u)  <  1' 

Remark:  In  the  M/M/1  queue  with  delayed  Bernoulli  feedback  with 
hQ(J^»j2)  "  q»  the  sufficient  conditions  become 


_A_ 

wq 


< 


1 


and 


A(p  +  A/p) 
a(q  -  A/p) 


<  1. 


These  conditions  are  slightly  more  conservative  than  the  necessary  and 
sufficient  conditions  in  Jackson  (1957)  which  are 


—  <  1  and  <  1 . 
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Proof.  We  use  one  of  Foster's  criteria  (cf.  Cohen  (1969)  p.  25) 
for  the  ergodicity  of  a  Markov  chain.  The  condition  is  that  ve  find 
aoae  nonnegative  function  f  such  that 


|M'<W  I  VI  <  - 

and  for  all  but  finitely  many  states  X°, 

El£<Cl>  -  £«»>  I  <1  *  «• 

Since  X°  •  (i^,  i2,  i^) ,  let 

f(X°)  -  f(ix,  i2,  i3)  "  +  i2  +  i3* 

Thus  f(X°)  is  the  total  number  in  the  system.  Now, 

•  <wv>  ■ 

Hence  if 


C  x 

»  -  VVV  if  h  >  °* 

+  A+a  «  Vh-h» 

if  i.  ■  0  and  >  0. 

V  L 


W  “  h0^l’^2^ 


<  0 


and 


X+a  (1  +  y  “  h0^1’^2^  +  X+a  "  h0(;Jl,J2^  <  °» 

and  since  lE[f  (X°  ^) |X°1  |  <  ",  X°  is  ergodic.  The  above  simplify  to  the 
stated  conditions.  a 
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An  example  of  an  application  of  Theorem  2.7  is  the  M/G/l  queue 
with  delayed  feedback  in  which 

3l  J2 

hl^l’  ^2’  "  hl^l’  ^  "  P1  p2  »  0  <  pi»  p2  <  1* 

Using  Theorem  2.7,  a  sufficient  condition  for  ergodicity  is  that 


—  <  1  and  j.  — r-  <  1 

W  a(l  -  p) 

where  p  ■  X/p.  This  can  be  shown  since  there  exists  J  such  that  for  all 
<Jr  J2)  with  Jx  +  j2  >  J 


A  <  _ A _ 

wh0(Jl.j2)  y[1  .  max(pi,p2)J] 


<  1 


and 


X(hi(ji,j2)  +  X/p)  HmaxCp^pj)  +  A/p) 

a(h0(jl’V  “  */v>  o(1  .  max(pi,p2)J  -  x/p) 


<  1. 


It  is  interesting  to  note  that  the  sufficient  conditions  do  not  depend 

on  the  specific  values  of  p^  and  p2-  In  fact  the  result  does  not  even 

depend  on  the  geometric  form  of  h^(j^,j2)-  The  only  necessary  property 
is  that  for  each  e  >  0,  there  exists  a  finite  number  of  states  (j^,j2) 
such  that  h^(j^,j2>  >  1  -  e. 

By  analogy  to  Jackson  networks  and  M/G/l  queues,  one  might  guess 
that  a  necessary  and  sufficient  condition  for  ergodicity  is  that 

X  .  ,  Xhl(Jl,J2)  . 

for  all  but  a  finite  number  of  states.  In  the  previous  example  this 
would  reduce  to  X/p  <  1.  Thus  an  Interesting  question  is  whether  there 

exists  a  system  with  X/p  <  1,  for  each  e  >  0,  has  only  a  finite 

number  of  states  with  h1(J1> J2)  >  1-e,  and  that  fails  to  be  ergodic. 


A  busy  cycle  is  comprised  of  an  Idle  period  followed  by  a  busy 
period.  During  Che  Idle  period  both  Q,  and  Q2  are  Idle.  During  Che 
busy  period  at  least  one  of  the  two  queues  has  a  customer  In  service. 

The  busy  period  Is  the  sum  of  i  +  j  Independent  random  variables,  1 
service  times  at  with  distribution  G  and  j  idle  times  at  Q2  which 
are  exponentially  distributed  with  parameter  X  +  a.  We  will  be  able  to 
examine  the  busy  cycle  by  filtering  the  (X°,  T°)  process  analyzed  in 
section  one.  Let  D  ■  {(0,  0,  0)}  and  E«NxMx{0,  1}.  Assume 
X°  -  (0,  0,  0)  and  NQ  -  0.  Let 

N  -  inf {1  I  X.  t  D,  1  >  N  ncW-  {0}. 

n  l  n-i 

Now,  we  define 

X  -  X?  and  T  -  T°  ,  n  e  W  . 
n  N  n  N 

.  n  n 

Theorem  3.1.  (X,  T)  is  a  Markov  renewal  process. 

Proof.  The  result  follows  from  Theorem  10.1.13  of  Cinlar  (1975). 

In  general,  the  busy  cycle  of  a  queueing  network  is  not  neces¬ 
sarily  a  renewal  process.  For  this  network,  however,  we  have  the 
following: 

Corollary  3.2.  T  ■  {T^ ;  i  e  K  }  is  a  renewal  process. 

Proof .  (X,  T)  is  a  Markov  renewal  process  whose  state  space  has 

only  one  point.  By  Theorem  10.1.11  of  Cinlar  (1975),  T  is  a  renewal 
process .  ° 

A 

Since  T  is  a  renewal  process,  we  need  only  obtain  the  distribu¬ 
tion  of  an  arbitrary  interval  to  characterize  the  process.  First,  we 
partition  the  semi-Markov  kernel  of  (X,  T) .  Let  D'  *  E  -  D  and 
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Theorem  3.3.  The  distribution  of  the  busy  cycle  is  given  by 

%D  +  %D'  *  ^  >  %'D'^  *  %'D  * 
n-0 

where  *  denotes  convolution  (cf.  finlar  (1975X10.3.1)  p.  323). 

Proof .  Follows  directly  from  theorem  8.15  of  Qinlar  (1969). 

We  could  analyze  Z^,  the  joint  queue  length  process,  as  a 
regenerative  process  with  T  as  the  set  of  regeneration  points.  However, 
it  is  easier  to  describe  the  behavior  of  Zt  between  output  points  (as 
done  in  section  two)  than  between  departure  points  leaving  an  empty 
system. 


4,  Customer  Flows 

In  section  four,  we  analyze  the  flow  of  customers  on  the  arcs  of 
the  network.  The  flow  on  each  arc  will  be  represented  as  either  a  point 
process  or  as  a  marked  point  process.  (See  Definitions  II. 3.1  through 
II. 3.4  for  these  concepts).  Recall  from  section  III. 3  that: 

N*  is  the  arrival  process, 

N*  is  the  input  process, 
o 

N  is  the  output  process, 
is  the  departure  process, 
is  the  feedback  process, 

Nr  is  the  reentry  process. 


Equivalently,  we  have 


Ta  is  time  of  the  nth  arrival, 
n  ’ 

T*  is  time  of  the  nth  input, 

T°  is  time  of  the  nth  output, 

is  time  of  the  nth  departure, 

is  time  of  the  nth  feedback, 
n 

is  time  of  the  nth  reentry. 

First,  we  examine  the  customer  flows  as  marked  point  processes. 

A  marked  point  process  (sometimes  called  a  labelled  point  process  or  a 

point  process  with  an  ancillary  variable)  can  be  thought  of  as  a  point 

process  in  which  each  point  T  has  an  associated  random  variable  X 

n  n 

lying  in  the  mark  space.  Marked  point  processes  are  useful  in  represent¬ 
ing  an  arrival  process  to  a  queue  with  several  types  of  customers.  The 

basic  point  T  would  be  the  time  of  the  nth  arrival  and  the  associated 
n 

random  variable  X^  would  be  the  type  of  the  customer.  Clearly  the  process 
(X°,  T°)  analyzed  in  section  one  is  a  marked  point  process.  Our  first 
result  is  a  restatement  of  Theorem  1.1  and  characterizes  (X°,  T°) ,  the 
marked  output  process. 

Theorem  4.1.  The  marked  point  process  (X°,  T°)  is  a  Markov 
renewal  process. 

Now  define 

Xn  "  V-  -  <*•  °>»  xd  “  (X„}, 

n 

X*  -  ZTf_  -  (1,  0),  Xf  -  {X*}. 

Q 

We  can  now  characterize  the  marked  departure  process  (X1*,  Td)  and  the 
marked  feedback  process  (X^,  T1). 

Theorem  4.2.  The  marked  point  process  (X^,  T^)  is  a  Markov 
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renewal  process  with  state  space  N  x  K. 

Proof.  Let  D  -  {(ir  i2>  0)  |  (i1,i2,0)  e  E}.  (Xd,  Td)  can  be 
obtained  by  filtering  (X°,  T°)  with  D  (i.e.  at  departure  points).  By 
Theorem  10.1.13  of  Qinlar  (1975;  p.  315),  (Xd,  Td)  is  a  Markov  renewal 
process  with  state  space  H  x  V  x  (0).  Clearly  we  can  allow  W  x  W  to 
be  the  state  space.  □ 

Theorem  4.3.  The  marked  point  process  (X^,  T^)  is  a  Markov 
renewal  process  with  state  space  W  x  K. 

Proof.  The  proof  is  the  same  as  the  proof  of  Theorem  4.2  except 
that  D  -  {(ir  i2,  1)  |  <ir  i2,  1)  e  E}.  O 

The  semi-Markov  kernels  for  (X^,  T^)  and  (Xd,  Td)  can  be  obtained 
from  the  semi-Markov  kernel  of  (X°,  T°).  Thus,  we  can  always  character¬ 
ize  (X°,  T°) ,  the  marked  output  process;  (Xd,  Td),  the  marked  departure 
process;  and  (Xf,  Tf),  the  marked  feedback  process,  as  Markov  renewal 
processes  with  a  countable  state  space.  The  result  does  not  follow  for 
the  marked  input  process,  (Xd,  T^);  the  marked  reentry  process,  (Xr,  Tr) 

or  the  marked  arrival  process,  (Xs,  T3) .  For  example,  let  have  a 

i  T  a 

deterministic  server.  X  ,  X  ,  and  x  will  not  have  the  Markov  property 
at  inputs,  reentries,  or  arrivals  unless  the  state  space  has  the  forward 
or  backward  service  time  of  the  customer  in  service  at  Q^.  But  this 
requires  an  uncountable  state  space. 

Queueing  networks  are  often  analyzed  by  decomposing  the  network 
and  analyzing  each  component  separately.  The  arrival  process  to  each 
component  is  assumed  to  be  a  renewal  process  or  a  Poisson  process.  Since 
the  arrival  process  to  a  component  may  be  the  departure  process  from  some 
other  component,  the  arrival  process  may  not  be  a  Poisson  or  even  a 
renewal  process.  It  would  be  useful  to  know  conditions  for  the  customer 
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flow  Co  be  weakly  lumpable  Co  Poisson  or  renewal  processes. 

The  Cerm  weakly  lumpable  has  been  used  by  Kemeny  and  Snell  (1960) 
in  connecdon  wlch  Markov  chains.  Serfozo  (1969)  defined  weak  lumpab- 
ilicy  analogously  for  Markov  renewal  processes.  Simon  (1978)  has  done 
furcher  research  in  analyzing  equivalances  between  Markov  renewal 
processes.  Our  deflnicion  of  weak  lumpabilicy  relates  marked  point 
processes  with  renewal  processes. 

Definition  4.4.  A  marked  point  process  (X,  T)  is  weakly  lump- 
able  to  a  renewal  process  iff  T  is  a  renewal  process.  If  in  addition 
T  is  a  Poisson  poinc  process,  then  (X,  T)  is  weakly  lumpable  to  a  Poisson 
point  process. 

For  example,  the  departure  process  from  an  M/M/1  queue  can  be 
modelled  as  a  Markov  renewal  process  (X,  T)  where  Tn  is  the  time  of  the 
nth  output  and  X^  is  the  number  in  the  system  at  time  (see  Disney 
et  al.  (1972)).  It  is  well  known  that  if  the  distribution  of  Xq  is 
the  equilibrium  distribution  of  an  M/M/1  queue,  then  T  is  a  Poisson 
point  process.  Thus  in  equilibrium,  (X,  T)  is  weakly  lumpable  to  a 
Poisson  point  process. 

For  a  Markov  renewal  point  process  (X,  T) ,  the  conditions  for  T 
to  be  a  renewal  process  can  be  expressed  in  terms  of  v,  the  distribution 
of  Xq,  and  Q,  the  semi-Markov  kernel  of  (X,  T) .  Note  that  a  Markov 
renewal  process  (X,  T)  may  be  weakly  lumpable  for  some  values  of  v  and 
not  for  others. 

Proposition  4.5.  (X,  T)  is  weakly  lumpable  to  a  renewal  process 

iff  for  all  n,  t^,t2»....tn 


37 


vQ(t1)Q(t2) ’ ’ *Q(tn)u  *  vQ(t1)uvQ(t2)u> • *vQ(tQ)u  (2) 


where  u  is  a  column  vector  of  ones  and  the  row  vector  v  is  the  distribu¬ 
tion  of  Xq. 

Proof .  Since  Tq  <_  £  • • • ,  T  is  a  renewal  process  iff 

T.  -  Tn,  T0  -!,,•••» T  -  T  , ,  are  i.i.d.  random  variables.  That  is 
1  0’  2  I  n  n-1 

for  all  n 


P  {T,  -  T_  <  t, , . . .  ,T  -  T  ,  <  t  }  »  IT  P  (T.  -  T„  <  t.}. 
v  1  0—1*  n  n-1  —  n  v  1  0  —  i 


But  this  is  (2) . 


o 


Simon  (1979)  has  proven  two  useful  results  on  the  weak  lumpability 
of  an  irreducible  Markov  renewal  process  to  a  renewal  process.  Assume 
that  the  Markov  chain  X  has  a  stationary  probability  distribution  ir. 

First,  the  only  renewal  process  which  is  a  candidate  has  interrenewal 
distribution  irQ(t)u.  Second,  if  (X,  T)  is  weakly  lumpable  to  a  renewal 
process  when  Xq  has  distribution  v,  then  (X,  T)  is  also  weakly  lumpable 
to  the  same  renewal  process  when  Xq  has  distribution  it.  In  summary,  if 
(X,  T)  with  initial  distribution  it  is  not  weakly  lumpable  to  the  renewal 
process  with  interrenewal  distribution  7rQ(t)u,  then  (X,  T)  cannot  be 
lumped  to  any  renewal  process  regardless  of  the  distribution  of  Xq. 

The  following  result  shows  that  under  most  conditions  of  interest 
the  only  possible  renewal  departure  process  is  a  Poisson  process  with 
rate  X. 

Theorem  4.6.  Assume  that  is  an  irreducible  Markov  chain  with 


a  stationary  probability  distribution  tt  and  that  Z£u  -  ZqU  (the  differ¬ 
ence  between  the  total  number  of  customers  at  time  t  and  at  time  0) 
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converges  in  dlstribucion  Co  an  almost  surely  finite  random  variable. 

If  (X^,  T^)  is  weakly  lumpable  to  a  renewal  process  then  (X^,T^)  is 
weakly  lumpable  to  a  Poisson  process  with  rate  X. 

Proof .  Assume  that  Xq  has  distribution  tt,  where  ir  is  the 
invariant  probability  distribution  for  the  Markov  chain  X^.  From 
Simon's  second  result  we  know  that  if  (X^,  Td)  is  not  weakly  lumpable 
with  distribution  ir,  it  is  not  weakly  lumpable  for  any  initial  distribu¬ 
tion.  Recall  that  X^  -  (Z^,  0).  Now 

Ztu  -  ZqU  =■  N*  -  N^  .  (3) 

Berman  (1978,  Theorem  7.4.2)  has  shown  that  if  the  moments  of  the 
Interrenewal  times  characterize  the  interrenewal  distribution  of  and 
if  the  left  hand  side  of  (3)  converges  in  distribution  to  an  almost 
surely  finite  random  variable  then  a  necessary  condition  for  N^  to  be  a 
renewal  process  is  that  it  have  the  same  interrenewal  distribution. 

Since  an  exponential  distribution  is  characterized  by  its  moments  we 
conclude  that  the  only  possible  renewal  departure  process  is  a  Poisson 
process.  □ 

In  the  remainder  of  this  section  we  will  restrict  our  attention 
to  systems  satisfying  the  hypothesis  of  Theorem  4.6.  Hence  determining 
if  the  departure  process  is  a  renewal  process  is  equivalent  to  determin¬ 
ing  if  the  departure  process  is  a  Poisson  process. 

Now  we  also  restrict  our  attention  to  queues  in  which  the 
probability  of  a  customer  feeding  back  is  independent  of  the  customer’s 
sojourn  time.  That  is. 


hg(j^, j2»x)  m 


Define  a  new  random  variable  V  to  be  the  remaining  service  time  of  the 
customer  in  service  at  time  t  in  Q^.  If  is  empty  define  V  *  0. 

Now  (Z,  V)  »  {Zt,  V  }  is  a  Markov  process  where  Z£  is  the  joint  queue 
length  at  time  t.  Let  p  be  the  Invariant  and  initial  probability  mea¬ 
sure  for  (Z,  V). 

Kelly  (1976)  has  called  a  network  quasi- reversible  if  (a)  depar¬ 
tures  of  group  i  customers,  for  i  *  1,2 . 1  form  independent  Poisson 

processes;  and  (b)  the  state  of  the  network  at  time  t  is  independent 
of  departures  from  the  network  up  until  time  t.  In  our  system  we  have 
one  type  of  a  customer,  i.e.  1*1.  The  state  of  the  network  at  time  t 
is  (Z  ,  V  ) .  Thus  condition  (b)  requires  (Z  ,  V  )  and  (N^;  s  <_  t}  to 
be  independent.  Melamed  (1979a)  has  introduced  a  weaker  condition 
called  pointwise  independence  which  requires  (Zt,  V£)  and  to  be 
independent  for  all  t. 

Theorem  4.7.  If  is  a  Poisson  process  and  (Zt,  V£)  is  inde¬ 
pendent  of  the  departure  process  up  to  and  including  time  t  (i.e.  of 
{N  ;  s  £  t})  then  Q1  must  have  an  exponential  server. 

Remark:  If  the  hypothesis  of  Theorem  4.7  is  satisfied  then  the 
network  is  said  to  be  quasi-reversible  (cf.  Kelly  (1976)).  The 
hypothesis  also  implies  pointwise  independence,  though  the  converse  is 
not  necessarily  true.  Melamed  shows  that  in  a  countable  state  regular 
Markov  process,  pointwise  independence  implies  that  is  a  Poisson 
process,  which  is  reassuring  even  though  not  immediately  applicable 
unless  the  server  at  has  a  special  structure  (e.g.  Erlangian) . 

Proof .  The  proof  relies  on  an  important  result  appearing  in  the 
following  chapter  on  general  networks.  Consider  a  non-anticipating 


Poisson  arrival  process  (i.e,  a  Poisson  arrival  process  in  which  the 
time  until  the  next  arrival  is  independent  of  the  current  state  of 
the  network);  then  the  distribution  of  the  state  of  the  network  imbedded 
before  arrivals  is  the  same  as  the  distribution  of  the  state  of  the 
network  at  an  arbitrary  point  in  time. 

Thus  in  our  network  the  stationary  distribution  of  (Z,  V) 
imbedded  just  before  arrivals  is  y,  the  invariant  probability  measure 
for  (Z,  V).  Now  consider  the  behavior  of  the  network  in  reverse  time. 
The  departure  process  after  a  time  reversal  becomes  the  arrival  process. 
Our  hypothesis  implies  that  the  time  since  the  last  departure  before 
time  t  is  independent  of  ( Z ,  Vt) .  But  after  a  time  reversal  this  is 
the  time  until  the  next  arrival  after  time  t.  Hence  the  departure 
process  becomes  a  non-anticipating  Poisson  arrival  process  after  a 
time  reversal.  Thus  we  conclude  that  the  stationary  distribution  of 
(Zt,  V£)  imbedded  just  after  departures  (i.e.  in  reverse  time  just 
before  arrivals)  is  also  y. 

Now  just  after  a  departure,  if  V  >  0  then  V  =  for  some  n 

t  t  n 

(i.e.  V  is  an  entire  service  time).  At  an  arbitrary  point  in  time, 

if  V_  >  0  then  V„  is  the  forward  recurrence  time  of  .  Since  they 
t  t  n  J 

have  the  same  distribution,  must  be  exponentially  distributed  (cf. 

Cinlar  (1975)  pp.  306-307). 

In  the  hypothesis  of  Theorem  4.7  we  require  (Zt,  Vfc)  to  be 
independent  of  (Ng;  s  £  t}.  We  think  this  is  a  reasonable  hypothesis, 
i.e.  it  holds  in  most,  if  not  all,  cases  in  which  the  M/G/l  queue  with 
delayed  feedback  has  Poisson  departures.  The  reason  follows  from 
Conjecture  4.9. 


Definition  4.8,  The  departure  process  is  uniquely  weakly  lump- 
able  to  a  Poisson  process  if 


P  {T“  <  t. , . 

W  1  —  1 


,T“  <  Cn} 

n  —  n 


P  {T^  £  t^ .  in 


n  -Xt 

it  (1-e  *) 

i“l 


implies  p  ■  y,  where  p  and  y  are  initial  distributions  for  (Zq,  Vq)  and 
p  is  the  invariant  probability  distribution  of  (Z,  V). 

Intuitively,  the  definition  says  that  if  the  network  is  not 
initially  in  equilibrium  then  the  departure  process  is  not  a  Poisson 
process. 

The  following  conjecture  would  resolve  the  entire  issue. 

Conjecture  4.9.  The  M/G/l  queue  with  delayed  feedback  is  unique¬ 
ly  weakly  lumpable  to  a  Poisson  process. 

There  is  evidence  supporting  Conjecture  4.9  but  we  have  been 
unable  to  prove  it. 

Proposition  4.10.  Unique  weak  lumpability  to  a  Poisson  process 
implies  quasi-reversibility  which  in  turn  implies  pointwise  independence 

Proof.  Assume  that  the  system  is  uniquely  weakly  lumpable  to  a 
Poisson  process.  Assume  that  (Zq,  Vq)  has  distribution  p,  hence  is  a 

Poisson  process.  Thus  at  any  time  t  >  0,  (Zt>  Vt)  has  distribution  p 
independent  of  {N^;  0  £  s  <_  t),  for  otherwise  would  not  have 

a  Poisson  distribution  by  unique  weak  lumpability.  Hence  the  system 
is  quasi-reversible.  As  noted  earlier  quasi-reversibility  implies 
pointwise  independence.  □ 

By  looking  at  the  contrapositive  to  Proposition  4.10  we  obtain 


the  following  result. 


Corollary  4.11.  If  the  system  is  not  quasi-reversible  and  if  in 

equilibrium  the  departure  process  is  a  Poisson  process  then  there  are 

two  distinct  (and  hence  an  uncountable  number  of)  initial  distributions 

for  (Z,  V)  yielding  a  Poisson  departure  process. 

Proof .  Let  T  denote  the  set  of  initial  distribution  for  (Z,  V) 

such  that  is  a  Poisson  process,  per.  If  the  system  is  not  quasi- 

reversible  then  the  system  is  not  uniquely  weakly  lumpable.  Hence  there 

exist  y  f  y  such  that  y  t  T.  Alternatively  if  the  system  is  not  quasi- 

reversible  there  exists  t  and  B  e  s  <  t)  such  that 

s  — 

P(B)  >  0, 

V(Zt’  v  e  c  I B}  *  ^C>-  <«) 

If  (4)  does  not  hold  for  some  t  and  B  then  the  system  would  be  quasi- 
reversible  since  (Zt>  V^)  and  ;  s  <  t}  would  be  independent.  Let 

V(zt*  V  E  c  I B}  =  y(c)- 

Then  y  e  T.  Once  we  have  two  elements  p,  y  e  T,  then 

py  +  (1  -  p)p  e  r, 

for  all  p  such  that  py  +  (l-p)p  is  a  probability  measure,  py  +  (l-p)p 
is  certainly  a  probability  measure  for  all  p  c  [0,  1].  Thus  we  have  an 
uncountable  number  of  elements  in  r.  It  is  possible  to  generate  more 
elements.  Take  any  y  e  T,  t  and  B  e  s  <_  t),  with  P(B)  >  0.  Let 

VV  V  e  c ' B} "  n(c>- 

Then  n  e  T.  And  as  before,  any  probability  measure  expressible  as  a 
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linear  combination  of  elements  of  T  Is  also  in  r. 

In  Corollary  4.11,  "if  the  system  is  not  pointwlae  Independent" 
could  have  replaced  with  "If  the  system  is  not  quasi-reversible"  and 
the  result  would  follow.  Before  concluding  this  section  we  will  briefly 
discuss  the  weak  lumpability  of  the  output  process,  (X°,  T°),  to  a 
renewal  process. 

Proposition  4.12.  In  the  M/G/l  queue  with  delayed  Bernoulli 
feedback  if  N°  is  a  renewal  process  then  N°  is  a  Poisson  process  with 
rate  A/q. 

Proof ♦  If  N°  is  a  renewal  process  then  N ^  is  a  renewal  process 
since  is  obtained  from  N°  by  selecting  points  with  a  fixed  probabil¬ 
ity  q.  But  must  be  a  Poisson  process  if  it  is  a  renewal  process. 
Consequently  must  be  a  Poisson  process  since  no  other  renewal  process 
when  subjected  to  Bernoulli  filtering  yields  a  Poisson  process.  Clearly 
since  must  have  rate  A,  N°  must  have  rate  A/q.  O 

Furthermore,  if  it  could  be  established  that  the  M/G/l  queue  with 
delayed  Bernoulli  feedback  is  uniquely  weakly  lumpable  to  a  Poisson 
process  then  is  not  a  Poisson  process  since  the  network  must  be  a 
Jackson  network  (G  -  M) .  From  Melamed  (1979b)  it  is  known  that  the  flow 


on  a  non-exit  arc  in  a  Jackson  network  is  not  a  Poisson  process. 


CHAPTER  IV 


RESULTS  FOR  GENERAL  SYSTEMS 

In  Chapter  IV  we  look  at  some  results  for  more  general  systems 
than  the  M/G/l  queue  with  delayed  feedback.  The  notation  in  Chapter  IV 
and  the  rest  of  the  dissertation  are  distinct. 

The  term  Markov  process  will  be  restricted  to  continuous  time 
stochastic  processes  and  the  term  Markov  chain  will  be  restricted  to 
discrete  time  stochastic  processes.  We  show  that  under  certain  condi¬ 
tions  a  Markov  process  (X(t) :  t  >_  0}  and  an  imbedded  Markov  chain 

{X(Ta-):  n  >  0}  have  the  same  invariant  probability  distribution.  The 
n  — 

Si  3 

sequence  T^,  T2,...  forms  a  Poisson  process  with  rate  a,  but  is  not 

necessarily  independent  of  the  Markov  process  (X(t) :  t  >;  0}.  Our 

proof  uses  an  external  observer  who  periodically  samples  the  state  of 

{X(t):  t  _>  0}  at  times  T®,  Ta .  The  sequence  T®,  T®,...  also  forms 

3  3 

a  Poisson  process  with  rate  a  but  is  independent  of  T^,  T^,...  and 

{X(t):  t  j>  0}.  We  first  show  that  the  Markov  process  (X(t):  t  ^  0}  and 

the  Markov  chain  (X(Te):  n  >  0}  have  the  same  invariant  probability 

n  — 

0 

distribution.  Next  we  show  that  the  Markov  chain  (X(Tn):  n  >_  0}  and 

{X(Ta-):  n  >  0}  have  the  same  invariant  probability  distribution.  Hence 
n 

(X(t) :  t  >  0}  and  {X(Ta-):  n  >  0}  have  the  same  invariant  probability 
—  n  — 

distribution. 

The  last  section  of  the  chapter  is  devoted  to  applications  of 
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Che  above  resulCs  Co  several  areas  of  applied  probability  including 
queueing  necvork  cheory  and  sCochasdc  concrol. 


1.  Assumptions  and  Nocacion 

Lee  (ft,  ^T,P)  be  a  probablliCy  space  and  (E,  $  )  a  complete 
separable  meCric  space  where  (9  is  Che  o-algebra  of  Borel  sets  of  E. 

Let  X  ■  {X(t):  t  >  0}  be  a  strong  Markov  process  with  an  indecomposable 
state  space  (E,  $  )  over  (ft,  «^,P)  with  right  continuous  sample  paths 
having  left  hand  limits  a.e..  Furthermore  assume  that  X  has  a  temporally 
homogeneous  transition  function  Pt(x,A),  x  e  E,  A  e  $  ,  with  an  invari¬ 
ant  probability  measure  p  and  that  P^{X(t-)  t  X(t)}  *  0  for  all  t. 

We  use  Pt  to  denote  the  following  operator  induced  by  Pt(x,A)  on 
(E,  $  )  measurable  functions. 


>tf(x)  *  /pt(x. 


dy)f (y) . 


We  use  pf  to  denote  the  following  linear  functional  induced  by  u  on  the 
(E,  $ )  measurable  function  f(x). 


pf 


/• 


(dx) f (x) . 


Since  p  is  an  invariant  probability  measure  in  the  above  notation. 


pPtIA(x)  -  yIA  =  p(A) ,  A  z  $  ,  t  >  0, 

where  IA(x)  is  the  indicator  function  of  the  set  A.  We  define  one  more 
operator  Ua  which  is  called  the  a-potential  operator  and  defined  to  be 


U°f(x)  -  /  e"atP.f(x)dt  - 

JO 


I' e_at  h" 


dy)f(y)dt,  a  >  0. 


0  0 

The  sequence  of  external  observation  times  0  “  Tq  <_  _<  .  .  .  is 

a  Poisson  process  with  rate  a  and  is  independent  of  the  Markov  process 
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(X(t) :  t  >  0}.  Let 

&L  “  o((X(s);  s  <  t),  (N®;  s  <  t)) 
t  —  s  — 

0 

where  Ng  is  the  counting  process  associated  with  the  Poisson  process 
{T®:  n  _>  0}.  Intuitively,  the  o-algebra  contains  all  the  informa¬ 

tion  of  the  Markov  process  (X(s):  s  ^  0}  and  the  Poisson  process 

{T®:  n  >  1}  up  to  time  t.  Let 
n  — 

*.>• 

Similarly,  can  be  interpreted  as  containing  all  the  information  up 

to  but  not  including  time  t.  Also  define 

»  o(X(s) ;  s  <  t). 

The  a-algebra  contains  all  the  information  in  the  process  X  up  to 

time  t.  Clearly, 

%  <= 

and  X  is  a  Markov  process  adapted  to  either  or 

Let  0  *  Tq  _<  T®  •  •  •  be  a  sequence  of  stopping  times. 

3  2L 

Define  X  *  X(T  -),  n  <  0.  We  will  show  that  under  certain  conditions 
n  n  — 

(Xf:  n  >  0}  is  a  Markov  chain  with  the  same  invariant  distribution  as 
n  — 

0 

the  Markov  process  X.  Our  proof  uses  the  Markov  chain  {Xr:  n  >_  0} 

where  X®  *  X(T®).  As  stated  in  the  introduction  we  first  show  that  X 
n  n 

and  {X^:  n  0}  have  the  same  invariant  probability  distribution  and 
then  that  {X  :  n  >  0}  and  {X  :  n  >  0}  have  the  same  invariant  prob- 

Ti  —  n  —  * 


ability  distribution. 
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2.  (X(t) :  t  >  0  }  vs.  { Xe :  n  >  0} 

_ — _ n _ — 

Proposition  2.1.  {X^:  n  _>  0}  is  a  Markov  chain  with  single  step 

transition  function 


P(x,A)  -  aUaI^(x),  x  e  E,  A  e  <p. 

Proof.  For  each  n,  T®  is  an  stopping  time.  Since  X  has  the 

strong  Markov  property  (X(T®):  n  _>  1}  forms  a  Markov  chain. 

6  6 

Now,  -  Tr  is  exponentially  distributed  with  parameter  a  and 

is  independent  of  the  Markov  process  X.  Hence, 


P (x,A) 


t(x,A)dt 

PtIA(x)dt 


aU  IA(x) . 


The  following  theorem  states  that  the  Markov  process  X  has  invari- 
ant  probability  measure  u  if  and  only  if  the  Markov  chain  {Xr:  n  ^  0}  has 
invariant  probability  measure  p. 

Theorem  2.2.  pPtIA(x)  *  u(A)  4=^  uaUaIA(x)  *  u(A),  a  >  0. 

Proof.  (  ) 

r  r  * 

-at 


uaU  IA(x) 


°  I  e'at  /P  t(x> 


dy)IA(y)dt. 


By  Tonelli's  theorem  (Royden  (1964),  p.  234)  the  right  hand  side  becomes 


J  ate  J  u(dx)  J Pc(x,dy)IA(y)dt 


u(A)  . 


(■^=)  Since  uoU  IA(x)  -  u(A), 


p(dx) 


Hence, 


*  fo  e-  Jpt(x, 


dy)IA(y)dt  =  y(A) . 


j  e  at  a^i(dx)  y*Pt(x, 


dy)IA(y)dt 


j; 


e  atay(A)dt. 


By  the  a.s.  uniqueness  of  Laplace  transforms  (see  Feller,  Vol.  II, 
p.  433)  combined  with  the  right  continuity  of  Pt(x,A) 


J'v(dx)  Jpt(x, 


dy)lA(y)dt 


u  (A) . 


Thus  uP,.IA(x)  -  u  (A) , 

C  A  -J 

The  previous  theorem  can  be  intuitively  interpreted  as  stating 

that  in  equilibrium  the  distribution  seen  by  an  external  observer  is  the 

same  as  the  distribution  at  an  arbitrary  point  in  time. 

Theorem  2.2  has  been  proven  for  countable  state  Markov  processes 

by  Clnlar  (1975),  Chapter  8,  Theorem  5.21  and  for  standard  Markov  proces 

ses  by  Nagasawa  and  Sato  (1963).  A  standard  Markov  process  is  required 

to  be  quasi-left-continuous.  That  is,  if  {Tn>  is  an  increasing  sequence 

of  stopping  times  with  limit  T,  hen  X(Tn)  -*■  X(T)  a.s,.  It  is  easy 


I 

I 

i 

I 


to  construct  a  system  which  has  an  uncountable  state  space  which  is  not 


quasi- left-continuous.  For  example  an  M/D/1  queue  with  X(t)  *  (Q^,  Vt), 
where  Q£  is  the  queue  length  at  time  t  and  is  the  remaining  service 
time  of  the  customer  in  service  at  time  t,  is  a  Markov  process.  The 
state  space  is  uncountable  and 

TQ  -  inf { t :  Vfc  -  1}, 


T  -  inf  { t :  T  >  T  , ,  V  <  ■=■}, 
n  n  -  n— l  t  —  n 


T  »  lim  T  , 
n  » 

is  a  sequence  of  stopping  times  with  the  lim  X(T  )  >  X(T)  a.s.. 

n  n 

3.  (Xe:  n  >  0}  vs.  {Xa:  n  >  0} 
n _ — _ n _ — 

{Xn:  n  >_  0}  is  a  discrete  time  stochastic  process  imbedded  in  the 
Markov  process  {X(t):  *  >  0}.  We  would  like  {Xa:  n  >_  0}  to  be  a  Markov 
chain.  In  general  (X^:  n  0}  may  not  be  a  Markov  chain.  In  the  appen¬ 
dix  we  construct  an  example  in  which  (X^:  n  .>  0}  is  not  a  Markov  chain. 
The  following  criterion  gives  sufficient  conditions  for  {Xa:  n  ^  1}  to 
have  the  Markov  property. 

Criterion  3.1.  If 

P{X(Ta)  c  B  |  (X  ;  t  <  Ta)}  -  P{X(Ta)  e  B  |  X(Ta-)}  a.s. 
n  c  n  n  n 

for  all  Be  S,  then  P(X*^  e  A  |  Xa,  Xn_^,...}  “  P^n+1  e  A  I  Xn^' 

Proof.  Let  the  regular  conditional  distribution  PQ(B  |  x)  be  a 

version  of  P {X(Ta)  e  B  |  X(Ta-)  ■  x},  (Breiman  (1968),  p.  77), 
n  n 

Since  T8  is  a  stopping  time 

n  l 
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P{Xn+l  e  A  ^  Xn  “  V  ^n-l  “  Xn-1’,,,} 


Jp(X(T^+1-)  e  A  |  X(Ta)  «  y>  Pn<dy  |  xj 


P{X3  e  A  I  Xs  -  x  }. 
n+1  '  n  n 


Q 


Let  us  define  to  be  the  forward  recurrence  time  until  the  next 
Ta  after  time  t.  Thus, 

V*  -  inf {Ta:  Ta  >  t>  -  t. 
t  n  n 


Theorem  3.2.  If  {X  :  n  >  1}  and  {X:  n  >  1}  are  stationary 
-  n  —  n  — 

Markov  chains  with  indecomposable  state  spaces  and  if 
P{Va  <  x  |  -  1  -  e"“X,  x  >  0, 


(1) 


then  u  is  an  invariant  probability  measure  for  Xq  iff  y  is  an  invariant 


probability  measure  for  X3 


n 


Proof.  Let  T3  -  {Ta},  T®  *  {Te}  and  T  «  Ta  U  Te  -  {T_}  where 
■  n  n  n 


Tl  -  min  {Ta,  Ta}  and 


Tl  1  r2  -  *  *  *  * 

Equation  (1)  implies  that  Ta  is  a  Poisson  process.  Te  was  defined  to  be 

a  Poisson  process  and  is  independent  of  T3.  Hence  T  is  a  Poisson 

process  with  rate  2a.  Let  X  ■  X(T  -).  Note  that  X(Te->  *  X(Te)  a.s., 

n  ti  n 
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-  li*(A)  (2) 


-  ua(A),  (3) 

£16  6  6 
where  u  and  u  are  the  stationary  distributions  of  X  and  X  , 

n  n 

respectively.  Equation  (2)  and  (3)  follow  from  the  fact  that  a  station¬ 
ary  Markov  chains  with  an  indecomposable  state  space  is  ergodic.  Hence 
in  both  (2)  and  (3),  the  expression  inside  the  brackets  converges  a.s. 
to  the  unique  stationary  distribution  (Breiman  (1968),  Theorem  7,16). 

(2)  and  (3)  are  identical  except  that  I^,e(T^)  in  (2)  replaces  1^(1^) 
in  (3).  But  the  probability  of  a  particular  observation  point  being 
either  an  arrival  or  an  external  observer  conditioned  on  the  previous 
history  is  .5.  That  is, 

vv(v  • k  i  _>-w<y  ■ k  i  *v> " 

n 

Consequently,  ye(A)  ■  ya(A)  for  all  A  e  S.  o 

By  Theorem  2.2  y  •  ye.  Thus  we  have  under  the  conditions  of 
Theorem  3.2  that  if  any  one  of  X£,  X®  and  X®  has  stationary  distribution 


1/2  if  k  =  0,1, 

0  otherwise. 


lim 

n-xo 


£wv<v 

1-1 _ 

1  +  E  IT®(Ti) 
i-1 


and 


lim 


£VW<V 

i-1 _ 

1  +  t  V<V 

i-1 
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V  Chen  all  three  have  stationary  distribution  p. 

4.  Applications  of  Results 

Work  Sampling.  In  work  sampling  an  employee  is  observed  at 
random  points  in  time.  At  each  observation  point,  the  particular  task 
being  performed  by  the  employee  is  noted.  From  these  observations, 
the  proportion  of  time  an  employee  devotes  to  each  task  is  estimated. 
Under  the  assumptions  of  Theorem  2.2,  if  the  observation  points  form  an 
Independent  Poisson  process,  the  estimate  converges  a.s.  to  the  true 
value.  In  other  sampling  schemes,  even  other  renewal  processes,  it  is 
possible  to  obtain  a  biased  result. 

Poisson  Arrivals  to  Queueing  Networks.  One  important  application 
of  Theorem  3.2  is  to  the  distribution  of  the  state  of  a  queueing  network 
Imbedded  at  arrival  points.  Let  the  queueing  network  be  modelled  as  a 
Markov  process  (X(t):  t  _>  0}  and  T*  be  the  time  of  the  nth  arrival.  If 
the  arrival  process  is  a  Poisson  process  satisfying  (1),  then  under  the 
assumptions  of  Theorem  3.2.,  in  equilibrium  the  distribution  of  the 
state  of  the  queueing  network  at  an  arbitrary  point  in  time  is  the  same 
as  the  distribution  Imbedded  just  before  arrivals.  In  order  to  apply 
Theorem  3.2  it  is  not  necessary  that  all  the  arrival  processes  be  Pois- 
son  processes.  If  an  arrival  process  of  a  certain  type  of  customer,  or 
of  customers  to  a  certain  queue  form  a  Poisson  process,  we  can  imbed 
before  these  arrivals.  The  other  arrival  processes  need  not  even  be 
renewal  processes.  Theorem  3.2  is  applicable  to  finite  capacity  systems 
(e.g,  M/G/l/N) ,  In  finite  capacity  systems  it  is  necessary  to  imbed  be¬ 
fore  each  of  the  Poisson  arrivals  whether  it  is  blocked  or  not.  It 


should  be  pointed  out  that  even  very  complex  queueing  networks  can  be 

modelled  as  a  Markov  process  satisfying  the  restrictions  of  section  2. 

The  state  space  E  is  required  to  be  a  complete  separable  metric  space 
00 

which  would  allow  R  ,  infinite  dimensional  Euclidean  space.  Hence  even 

a  queueing  network  with  renewal  exogenous  arrivals  and  general  multi- 

server  queues  can  be  modelled  as  a  Markov  process. 

The  above  result  is  a  generalization  of  a  well  known  result  that 

in  equilibrium  in  an  M/G/l  queue  the  distribution  of  the  queue  length 

seen  by  an  arrival  is  the  same  as  the  distribution  of  the  queue  length 

at  an  arbitrary  point  in  time,  (Cooper  (1972)  p.  134), 

Stochastic  Control.  Let  Ta  be  the  nth  time  we  observe  the  system 
-  n 

to  be  controlled  and  based  on  the  observation  reset  some  control  para¬ 
meter.  For  example  consider  an  M/G/l  queue  which  we  are  attempting  to 
control.  Our  Markov  process  might  be  ®  (Qt,  Ut,  St)  where  Qt  is  the 
queue  length  at  time  t,  Ut  is  the  length  of  service  accumulated  by  the 

customer  in  service  at  time  t  and  St  is  a  parameter  controlling  the 

speed  of  the  server.  Let  Tr  be  an  independent  Poisson  process.  Define 

X3  *  XOT3-).  Based  on  Xa  we  take  some  action  on  the  system.  Several 
n  n  n 

possibilities  are  removing  or  adding  customers,  changing  the  service 
rate  or  turning  the  server  on  or  off.  We  incur  a  cost  for  each  obser¬ 
vation  and  action  taken.  In  addition  there  is  a  cost  associated  with 
the  length  of  time  in  each  state.  Our  objective  is  to  minimize  ex¬ 
pected  cost  per  unit  time.  Based  on  Theorem  3.2  we  need  only  solve  for 

the  invariant  distribution  of  Xa  in  order  to  determine  our  costs.  From 

n 

this  it  is  possible  to  determine  the  optimal  control  strategy  in  order 
to  minimize  expected  cost  per  unit  time. 


Poiason  Departure  Processes  from  Queueing  Networks.  If  a 
departure  process  from  a  queueing  netvork  is  a  Poisson  process,  is  the 
stationary  distribution  imbedded  just  after  departures  the  same  as  the 
distribution  at  an  arbitrary  point  in  time?  Theorem  3.2  appears  to  be 
inapplicable  since  (1)  is  violated.  That  is  the  time  until  the  next 
departure  is  clearly  dependent  on  the  current  state.  Let  T^  be  the  time 
of  the  nth  departure  and 

■  t  -  sup{Ta:  T*  <  t>. 

Thus  is  the  backward  recurrence  time  to  the  last  departure  before 
time  t.  Consider  the  reversed  Markov  process.  The  departure  times  are 
arrival  times  in  reverse  time.  If 


P{V^  <  x  i  (X(s) ;  »  >  t)}  -  1  -  e~“*  (4) 

then  the  time  since  the  last  departure  is  independent  of  the  future. 

Hence  in  reversed  time  (1)  holds.  If  the  reversed  Markov  process 

satisfies  the  conditions  of  section  2  and  {X(Ta+):  n  >  0}  is  a  stationary 

n  — 

Markov  process  with  an  indecomposable  state  space  then  (X(t):  t  ^  0}  and 

(X(T*+):  n  >  0}  have  the  same  invariant  probability  distribution.  It 
n  — 

should  be  pointed  out  that  (4)  may  be  satisfied  for  judicious  selections 
of  a  Markov  process  modelling  a  system  and  not  for  others.  For  example 
consider  an  M/M/1  queue  with  X^  -  (Qt>  A£)  and  X^  *  (Qt,  Bt)  where 
Qt  is  the  queue  length  at  time  t,  A£  Is  the  amount  of  service  accumu¬ 
lated  by  the  customer  in  service  and  Bt  is  the  remaining  service  time  of 
the  customer  in  service.  The  Markov  process  X^  satisfies  (4)  but  X ^ 
does  not.  In  X^  the  Markov  chain  Imbedded  after  departures  has 


3«S.  * 


A^,a+  ■  0  a. 3..  Thus  Xp^  does  not  have  the  property  that  the  distribu- 
n  c 

tion  of  the  state  of  the  network  imbedded  just  after  departures  is  the 

(2) 

same  as  the  distribution  at  an  arbitrary  point  in  time,  while  X^  does 
have  the  property  (in  equilibrium). 


j 

i 


CHAPTER  V 


DISCUSSION 

Davignon  (1977)  points  out  that  in  the  queue  with  instantaneous 
feedback,  published  research  reports  outside  of  the  computing  literature 
are  rare.  The  same  is  true  only  more  so  of  the  queue  with  delayed  feed¬ 
back.  In  the  computing  literature,  the  queue  with  delayed  feedback  is 
a  useful  and  frequent  model  of  the  behavior  of  a  computer  system.  How¬ 
ever  there  are  few  theoretical  results  beyond  expected  values.  The 
few  results  which  are  more  general  are  usually  special  cases  of  results 
on  Jackson  networks.  This  dissertation  analyzes  the  M/G/l  queue  in 
order  to  provide  results  for  practitioners  modelling  a  system  as  an 
M/G/l  queue  with  delayed  feedback  and  also  to  gain  insight  into  queueing 
networks. 

1.  Summary 

Chapter  I  Informally  described  the  topic  of  queues  with  feedback 
and  reviewed  pertinent  literature.  Chapter  II  formally  described  the 
M/G/l  queue  with  delayed  feedback  in  terms  of  the  governing  sequences  of 
random  variables.  The  governing  sequences  of  random  variables  are  the 
interarrival  times,  the  service  times  at  the  upper  and  at  the  lower 
queue,  and  the  probability  of  a  customer  feeding  back.  The  probability 
of  a  customer  feeding  back  was  allowed  to  depend  on  the  past  history  to 
a  limited  extent.  In  section  3  the  processes  of  interest  were  defined. 
The  processes  of  interest  are  (generally  complicated)  functions  of  the 


57 


governing  sequences.  The  processes  of  interest  are  the  flow  processes: 
arrivals,  inputs,  outputs,  departures,  feedbacks,  and  reentries;  and 
the  queue  length  process  in  both  continuous  time  and  imbedded  at  out¬ 
puts. 

In  Chapter  III,  we  analyzed  the  processes  of  Interest.  Our  first 

result  characterized  (X°,  T°),  the  system  at  output  points,  as  a  Markov 

renewal  process.  This  result  provided  the  foundation  for  much  of  the 

analysis.  The  subsequent  lemma  exhibited  the  semi-Markov  kernel  of 

(X°,  T°).  With  state  dependent  feedback,  the  possibility  of  a  reducible 

state  space  arises.  A  simple  necessary  and  sufficient  condition  for  the 

irreducibility  of  (X°,  T°)  was  determined.  Irreducibility  in  turn 

implied  that  (X°,  T°)  was  aperiodic.  Under  the  assumptions  of  the  model 

the  lifetime  L  ■  sup  T°  was  almost  surely  infinite.  The  above  conditions 
n  n 

on  <^X°,  T°)  could  be  easily  determined  from  the  governing  sequences. 
Ergodicity  was  more  difficult  to  verify.  In  order  to  determine  if  X°  was 
ergodic  it  was  necessary  to  determine  if  there  existed  a  stationary 
probability  vector  for  the  Markov  chain  X°.  However  an  easily  computed 
sufficient  condition  was  established  using  Foster's  criteria.  Thus  it 
is  easy  to  determine  whether  (X°,  T°)  is  an  aperiodic  irreducible 
Markov  renewal  process  with  an  infinite  lifetime.  In  addition  there  is 
an  easily  checked  sufficient  condition  for  ergodicity. 

Z,  the  continuous  time  queue  length  process,  was  analyzed  by 
using  results  about  (X°,  T°).  Z  was  shown  to  be  a  semi-regenerative 
process  with  semi-regeneration  times  T°. 

In  the  next  section,  the  busy  cycle  was  shown  to  be  a  renewal 
process  and  an  expression  was  found  for  the  distribution  function  of 
the  interrenewal  times. 


i 
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Section  4  analyzed  the  flow  processes  on  the  arcs.  In  particular, 
the  output  process,  feedback  process,  and  departure  process  were  shown 
to  be  Markov  renewal  processes.  It  was  also  shown  that,  under  general 
conditions,  if  the  departure  process  was  a  renewal  process  then  it  must 
be  a  Poisson  process.  A  conjecture  was  stated  which,  if  verified,  would 
in  turn  show  that  if  the  departure  process  was  a  Poisson  process, 
must  have  an  exponential  server.  Thus  the  only  M/G/l  queue  with  delayed 
Bernoulli  feedback  having  renewal  departures  would  be  the  M/M/1  queue 
with  delayed  Bernoulli  feedback,  which  is  a  Jackson  network.  This  would 
in  turn  imply  that  the  M/G/l  queue  with  delayed  Bernoulli  feedback, 
never  has  a  renewal  output  process. 

Chapter  IV  discussed  some  general  results  which  were  used  in 

Chapter  III  but  are  also  useful  in  areas  other  than  the  queue  with 

delayed  feedback.  It  was  shown  that  if  {T^}  is  a  sequence  of  stopping 

times  for  a  Markov  process  (X(t):  t  _>  0}  with  invariant  probability 

measure  p,  and  X(T  -)  is  a  Markov  chain,  then  p  is  an  stationary 
n 

probability  measure  for  X(Tn~) .  This  can  be  used  to  show  that  Poisson 
arrivals  to  a  queueing  network  see  the  same  distribution  as  at  an 
arbitrary  point  in  time. 

2.  Future  Research 

Several  areas  of  further  research  come  to  mind  as  extensions  of 
the  present  work.  Foremost  is  proving  or  disproving  Conjecture  4.9  on 
unique  weak  lumpabllity.  Verification  of  Conjecture  4.9  would  charac¬ 
terize  all  M/G/l  queues  with  delayed  Bernoulli  feedback  having  renewal 
departure  processes  or  renewal  output  processes. 

Another  interesting  question  is  whether  there  exists  a  system 


which  fails  to  be  ergodlc  but  has 


for  all  but  a  finite  number  of  states.  More  substantial  but  along  the 
same  lines  would  be  determining  necessary  and  sufficient  conditions, 
which  are  easily  computable,  for  the  system  to  be  ergodic. 

A  third  untouched  area  is  the  difficult  but  needed  study  of  the 
sojourn  time  in  queues  with  delayed  feedback.  In  computer  models  the 
sojourn  time,  i.e.  the  departure  time  minus  the  arrival  time  of  an 
arbitrary  customer,  is  often  the  process  of  interest.  This  distribution 
is  unknown  even  in  the  case  of  most  Jackson  (1957)  networks  (cf.  Simon 
and  Foley  (1979)).  In  particular  the  sojourn  time  distribution  is 
unknown  even  in  the  case  of  the  M/M/1  queue  with  delayed  Bernoulli 
feedback. 

3.  Closing  Comments 

Most  of  the  work  in  this  dissertation  and  in  queueing  network 
theory  in  general  involves  structural  results  and  characterizations  of 
processes.  There  is  a  need  for  accurate  approximations  and  tight  bounds 
on  the  processes  of  interest  to  supplement  the  structural  results.  Even 
more  strongly,  there  is  a  need  for  computable  accurate  approximations 


and  bounds. 


APPENDIX 


AN  EXAMPLE  IN  WHICH  (X*:  n  >  1}  IS  NON-MARKQVIAN, 

n  — 

Let  X  ■  (X(t):  t  >  0}  be  a  Markov  process  and  {T® j  a  >  0}  a 
—  n  — 

sequence  of  stopping  times  for  X.  We  construct  an  example  In  which 

{X®:  n  >  1}  fails  to  be  a  Markov  chain  where  X®  ■  X(T®-).  Our  state 
n  n  n 

space  E  Is  the  closed  interval  [-1,  1],  Let  2^,  Z^,  be  a 

sequence  of  1.1. d.  exponential  random  variables  with  parameter  a. 
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Fig.  4  -  A  Realization  of  X(t) 

The  points  of  discontinuity  not  on  the  zero  axis  form  a  Poisson 

A  a 

process.  At  a  point  of  discontinuity  t  not  on  the  zero  axis,  X(t  )  *  1 
if  X(t*-)  >  0  and  X(t*)  ■  -1  if  X(t*-)  <0.  At  a  point  of  discontinuity 
t*  on  the  axis  (i.e.  X(t*-)  ■  0),  X(t*)  -  1  if  X(t)  approaches  xero 
from  below  at  t*.  Similarly,  X(t*)  •  -1  if  X(t)  approaches  zero  from 
above  at  time  t*. 

X  »  tX(t):  t  >  0}  is  a  Markov  process  satisfying  the  assumptions 

in  the  first  paragraph  of  section  2  and  {Ta:  n  >_  1}  is  a  sequence  of 

stopping  times  for  X.  Let  Xa  *  X(Ta~).  To  show  that  X®  is  not  a  Markov 

n  u  ti 

chain  we  need  only  note  that 

P{xV  >  0  |  Xa  -  0}  -  .5(1  -  e'a) 

n+i  n 

while 

p(Ci » 0 1  <  •  °-  Ci  > 01  -  °- 

Note  that  Criterion  3.1  does  not  hold  for  the  above  process.  In  addition 
(1)  of  Theorem  3.2  falls.  It  is  conceivable  that  if  (1)  holds  then 
it*  n  >  1)  must  be  a  Markov  chain. 
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